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Introduction 


Along time ago, ina datacenter far, far away, Microsoft BizTalk Server became to life. It 
was back in 2000 when we saw it for the first time. Although it was not until version 2004 
when it really transformed the integration landscape using Microsoft technologies, it 
indeed changed my life completely 

"The book that you have in your hands itis a journey that starts by revealing what is 
under the hood of the product ata low level. Even though you have strong BizTalk Server 
Knowledge, d you not skip it, because it provides base knowledge for the rest 
of the chapters 

wrote Chapters 2 and 3 to explain almost everything I know about performance 
‘counters, analysis techniques, and publie tools that are related to the scope of the book. 

Chapter 4 is all about optimizing the BizTalk Server platform using essential 
‘configurations that you can apply to a BizTalk environment, proactively and reactively 

















you expecta compilation of how to solve all the issues you can ever face, you will get 
probably disappointed. The idea of the chapter is to create a solid foundation so that 

‘you will be able design a BizTalk Server application thinking ofa concept that I call: 
application priority levels, 

Especially for mission-critical solutions, and BizTalk Server usually handles them, 
troubleshooting and fixing bugs on time is essential, as these applications frequently 
require zero downtime. Therefore, efficient application instrumentation becomes crucial 
to reduce the time of finding issues and to start working in the bug as soon as possible. 
"This topic is explained in Chapter 5, where you will learn how to implement Event Trace 
Tor Windows and to leverage in the Business Activity Monitoring feature to log important 
events and data related to the low. 

Chapter 6 will drive you to the most common actions you can take asa developer to 
‘make sure your applications will help the production server run them smoothly. You will 
learn how to improve your schemas definitions, maps, orchestrations, and pipelines. As 
‘we consider that custom pipelines components are essential for low latency scenarios 
(among others), you will also learn how to develop custom pipeline components 

Chapter 7 explains how take advantage ofa Microsoft NET Framework feature called 
side-by-side versioning, You will lam how to use it in your BizTalk Server projects to 
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reduce downtime at maximum, Additionally, it provides an introduction to busines 
rules and performance recommendations related toit. You might wonder why Ihave 





decided to include business rules. Well, although i's nota topic explicitly related to 
performance, it provides huge benefits in terms of reducing application downtime and. 
‘orchestration complexity 

A mature BizTalk Server infrastructure should be able to proactively monitor every 
aspect of BizTalk Server and for that propose I have decided to write about a fantastic 
tool called BizTalk 360, This is all detailed in Chapter à. 

Successfully designing and evolving a BizTalk Server platform are tasks that 
require including unit and performance testing procedures as part or the application 
development lifecycle, Chapter dives into these topies using Microsoft Visual Studio 
Testing features. 

Finally, Chapter 10 closes the journey with real-life scenarios explained through a 
Betitious company called ACME corporation, 

hope you learn as much reading the book as I did while writing it 





Who Is This Book For? 


Ifihisis the first contact you've had with BizTalk Server, Lam not going to lie to you—you 
are most likely going to get lost. This book assumes that you have already the following 
knowledge: 


+ BizTalk Server administration and troubleshooting exper 





+ Strong development NET Framework experience 
+ Familiarity with Microsoft Windows Communication Foundation 


You will not learn about the BizTalk Server basie stuff No. You will learn about the 
techniques that make the difference. 
you are passionate about BizTalk Server and performance, 





does not matter ifyou 
are an architect, administrator, developer, or a TV presenter. This book is for you! 


Requirements and Source Code 


Along the book you will find several walkthroughs that contain step-by-step instructions 
to develop BizTalk Server solutions that are relevant to the discussed topic. The source 
code is available on GitHub via the book's product page, located at waw.apress .con/ 
978-1-4842-3993-3. 

"To compile the code, you need the following system setup: 








+ Microsoft BizTalk Server 2016 Developer edition—The full 
installation is recommended, as you will create solutions with BAM 
as well (no need to install EDI as this book does not caver it). The 
Enterprise edition of the product will work also. 


+ Microsoft Visual Studio Enterprise edition.— Chapter? requires you 
tw have this edition because is all about testing features and those are 
not available in the Microsoft Visual Studio version. The Community 
edition is not supported. 

^ Microsoft Windows 10 or Microsoft Windows Server 2016 
(preferred) Windows 10 is fine for running the code, but ify 
want to practice some ofthe optimizations detailed in Chapter 4, you 
must have Windows Server 2016. 








+ Microsoft Internet Information Server There is a lab in Chapter 9 
that deploys a WCF service, so you will need to install IIS in your 
machine. 


BizTalk Server 2016 bas several requirements that must be in place as well L 
recommend that you read the following Microsoft documentation: 
https://docs.nicrosoft. con/en-us/biztalk/ install-and-config-guides/ 
hhardvare-and-software-requirenents-for-biztalk-server-2016 
Good luck! 
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CHAPTER 1 


Revealing the Black Box 


Most of the people who start their journey with BizTalk Server are usually not aware of 
how BizTalk Serv process, and sends messages, They instinctively hellev 
that it is something that just happens, driven by a mysterious hidden force. While 
atthe beginning this can be something even pleasant, when you deploy a solution 
to production and problems arise, troubleshooting BizTalk Server without knowing 
how the engine works can create confusion, delays, and in the worst of 
frustration. During my career as a premier fied engineer, have assisted customers 
with several critical situations, and you would be surprised how many times I heard 
the sentence: 

“What is happening with this black box! I do not understand i! 

In this chapter, you learn how BizTalk Server works from an internal point of view. 
"This topic is crucial if you want to become an expert BizTalk Server developer or a 
solution architect, but also ifyou want to master the tuning techniques required to adjust 











your platform to every situation. The chapter reviews the following topics: 


* Bs 





ial XML principles and how BizTalk Server uses th 





^o How BizTalk Server works internally 
^o Hosts and host instances 

^o Subscriptions 

+ Messages 

+ Publishers 

^o Subscribers 

^ The Message Box database 


+ Publication and dequeue processes 
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CHAPTER! REVEALING THE BLACK BOX 


Ifyou are a seasoned BizTalk Server user, you might notice some content 
missing here, such as tracking and BAM. Although these topics are important from a 
functionality point of view, they have been moved to further chapters af the book for 
darification and simplification proposes. This chapter focuses on the pieces of the 
engine that BizTalk Server uses to process messages. Do not worry, the book covers 
tracking and BAM functionalities in detail in Chapters 4 and 5. 

Having a solid foundation of the topics discussed in this chapter allows you to create. 
robust BizTalk Server solutions. Hopefully, you will never say again: “What is happening 
‘with this black box!” 


XML Language Premier 


Although the book assumes you have a strong knowledge of XML, this section covers all 
the XML topics that the engine uses to operate. 

BizTalk Server works with messages encoded in XML format. The W3C (World Wide 
Web Consortium) provides rules and guidelines for creating standard XML definitions. 
‘Schemas created by BizTalk Server follow this specification, so you can import BizTalk 
Server schemas to any other XML tools that follow this specification without too much 
effort (and vice versa). 

Ifyou want to master your XML skills, you could visit the World Wide Web. 


Consortium web page at http: //wi-w3. org/. 





XML Schema Concepts Used by BizTalk Server 


n about the XML elements that BizTalk Server uses to implement 







‘An XML attribute is a data container used to save additional information related to an 


XML element; they are like properties ofan element. Attributes can be associated with 





any ofthe simple data types but cannot be nested (elements can be nested when using 
jes though). Because af this restriction, you cannot create attributes as 
ves only as simple data types. 


complex ty 





complex ty 
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In the following example, Book is an element with a value of “La Perla Negra’ As you 
‘can see, the TSBN is the attribute ofthe Book element and has a value of 00078743649332. 


Book TSBN="00078743649332" sLa Perla Negrac/Book> 


Namespace 


BizTalk Server uses the concept of NaneSpace to avoid confusion between elements and. 
attributes names (see Figure 1-1). In this way, unique elements and attributes can be 
granted for the same schema definition and provided in an XML instance. In addition to 
this, the BizTalk engine uses this concept implemented as the targetNaneSpace property 
(INS) in several ways: 

+ To identify the message within the message box database 


+ To create subscriptions based on particular properties called 
promoted properties 


+ To publish the message along with the binary information. 





Figure 1-1. Example ofthe namespace definition 


Element 


Elements are the building blocks of an XML. They can behave as containers to hold text, 
elements, and attributes. Elements classified as simple or complex (see Figure 1-2). 
‘+ Simple: Simple data types (such as integer, DateTime, and string) 


+ Complex: Elements that contain other elements and attributes (such 
as include and import options) 
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Figure 1-2. Simple and complex elements 





Note Itis crucial to note that element names are case sensitive. That implies 
that, for the BizTalk Server engine, the element name price is different than 
Price (because the first character is uppercase). You should pay attention to this 
fact because if you provide the wrong element names, the BizTalk engine might not 
identify the message, or the initial validation can fal 








As shown in Figure 1-2, the next elements are simple: 
+ ISIN (string) 

+ Tithe tring) 

+ Pages (integer) 

+ Writtendate (DateTime) 


However, the record Writer is complex. 


BizTalk and XML Namespaces in Detail 


An XML document may contain elements or attributes rom more than one XML. 
definition (more than one namespace declared in the document). 
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For instance, Imagine that you are dealing with an XML document designed to 
define book information (see Figure 1-3). Every book has a field called ID that is used 
to identify the book. Similarly, there is an ID field to identify the writer of the book, as 
shown in Figure 1-3, 


Figure 1-3. XML definition example 


‘As you can see, the schema definition has two IDs. To avoid confusion, you can 
create two different schema definitions: The frst one defines the whole schema, and the 
second one defines only the writer structure (see Figure 1-4). 





Perla Wegrac/titie> 








oae writer cid 





— —— 
—E 
ES 
</sookintornet ion] 





Figure 1-4. Default and custom namespaces definition 


The Declaration 





BackInfornation xnlns="http://http://Books.con/BookInfornation" is what is 
called the default namespace, and it applies to all elements in the schema that do not 
ave a custom namespace definition. 

On the other hand, xnlns:writer="http://B0oks.con/liriter" represents a custom 
namespace. 

Notice that the writer data has the prefix ":kriter” and the default namespace does 
not have such a definition. Using this prefix later in the XML document allows you to 
identify all the elements in the custom namespace and not in the default one. As outlined 


in the previous example, the writer element has its definition, shown in Figure 1-5. 
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<WriterData> 
<writer:id>00000023</writer: id> 
<uriter:Nane>agustin</vriter:Nane> 
<uritør:LastName>Mantaras</writer:LastNane> 
</WriterData> 


re 1-5, Custom namespace writer data 
pa 


The custom namespace xnlns:writer-http: //Books .con/Writer defines the 
elements ofthe child record WriterData (Id, Name, and LastName). 

As BizTalk Server implements custom functionality that is nat used by any oth 
software, it can leverage the XML namespace technology to access this information, 
BizTalk Server adds the following two namespaces in every BizTalk Server schema: 








+ Target Namespace, When you create a new schema using the BizTalk 
Editor, a target namespace ls added by default (see Figure 1-6). The 
engine uses this information along with the root node name ofthe 
document to identify the message, The identification is implemented 
by the targetNaneSpace property of the schema and by default is 
constructed as shown here 


http://Visual StudioProjectNane.NaneOfTheSchena 


— 
-chescomplextvpe> 
— * 





Figure 1-6. Default name space definition. 
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tensions Namespace. Ifthe schema is representing a at file or an 
EDI schema, BizTalk adds a reference to this namespace: 


‘xnlns:bs"http://schenas.microsoft.com/BizTalk/2003" (see Figure 1-7) 








Figure 1-7. Annotation definition under a custom namespace 


BizTalk uses this prefix within the message to add the flat file extensions that contain 
delimiter and positional information in the form of annotations. 

"The elements within the annotation are used during the disassembling stage at the 
pipeline level to create the XML representation ofa specifie fat fle, 


Identification of an Incoming Message 


ngine extracts the default namespace 
(target namespace) and the main root node name. These values are internally 
concatenated (targetNanespace#rootNodeNane) and written in the context ofthe 
message as a property called Message Type. You will earn about message properties 
Tater in the chapter. 

"The Message Type must be unique so that the BizTalk Server engine can determine 
the correct schema and apply itto the received instance. Ifyou deploy schemas with the 
same message type, BizTalk Server always gets the same type (usually the one created 
in the fest place), and it ignores the rest. The consequences of this action could be very 
unpredictable because the engine could publish messages into the wrong host queue 
table, or raise a routing failure exception as subscribers might not exist for that message. 

We will dig into this topic later in the message engine section. 


When BizTalk receives a message, 





message 
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BizTalk Server Schemas 


A schema for BizTalk Server is like the definition of the message itself. It contains the 
specification by adding elements, attributes, types, complex types, and more. 

BizTalk schemas can be created by using third-party schema creation tools and the 
BizTalk Schema Editor, shipped with the produet and fully integrated into Microsoft 
Visual Studio 2015. Developers should consider the use of the BizTalk Editor tool as the 
preferred method for creating BizTalk Server schemas, since Mierosoft guarantees that 





the schemas created using this tool are fully supported. 


BizTalk Server Schema Types 


As discussed previously, BizTalk Server 2016 can natively process structured messages 
using the following formats: 


^o Fiat fle schemas 
+ XML schemas 

+ ISON (with alittle help of JSON pipelines) 
+ Fiat fle schemas 

‘There are two types of fa fle schemas: 


+ Delitnited—This is the case when a specifie character separates 
records or fields. 


+ Positior 





J The length ofthe elements iself identifies the feld In 
Other words, elements are constructed using the position in the file. 


Because public XSD by the W3C does not natively support the at file structure, 
BizTalk uses the annotation capabilities (lat file extensions) of the XML technology 
to provide all the required functionality. The example in Figure 1-8a shows fat fle 
annotations used in BizTalk schemas. 








Figure 1-8a. Flat file annotation 


a 
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Delimited Flat Files 


A delimited file contains one or more fields separated by a delimiter 





aracter The most 


‘common characters are: 
* Comma() 
+ Semicolon (CSV fles) 
* Pipe() 





However, you can work with any desired character Itis important to note that if you. 
setup a separator, that character should not appear as part of the text as BizTalk Serv 





‘considers that character a field separator. To overcome this problem, you can enclose the 
text that contains the separator using double quotes. 

For instance, consider the following flat fle: 
Agustin, Mantaras, Rodriguez, "BizTalk 2016, Developing" f« 

In this scenario, you can see four columns because the double quotes enclose the 
book title (see Figure 1-8b). 





gustin] wantaras] Rodriguez] "&izTaTk 2016, Developing" he 














Figure 1-8b. Flat file annotation 
‘On the other hand, look at this one: 
Agustin, Mantaras, Rodriguez, BizTalk 2016, Developing 1 


You can see five columns, as BizTalk considers all the comma characters as column. 
Figure 1-80). 





separators (s 








guetta] wamraras| Rodriguez] sizTaTk 2016] peveToping]t« 











Figure 1-8. Hat file annotation 
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Positional Flat Files 


Usually, inthis type of lat file, every line identifies a different record because at the end 
of the line there is an end-of-record character. This character is considered the delimiter 











character, and in most ofthe cases, tis represented by a carriage return (44). Fields ane 
Identified within that record using a fixed length. 

Look at the following message instance: 
Agustin Mantaras Rodriguez BizTalk 2016, Developing fe 


‘The length of the elements itself identify the individual columns within the record, as 
shown in Figure 1-8d. 


10 10 10 30 
[gustin —Fantaras Rodriguez BizTalk 2016, Developing P] 








Figure 1-8d. Flat file annotation 


Four columns with a fixed length of 10 characters and one with 30. You can see the 
record separator at the end (in this ease € 


XML Schemas 


An XML schema represents the definition of an XML message. XML documents are 
formed in a typed hierarchical structure that is defined by the schema. BizTalk uses 
schemas to validate the message and define the data associated with it. It is represented 
physically by an XSD file. 

Figure 1-9 shows the user-friendly view generated by the BizTalk Editor. 





Figure 1-9, Visual XML representation 
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Figure 1-10 shows the pure XML representation, 





Figure 1-10. XML representation 


JSON Messages 


Although JSON messages are not implemented using XML technology, I decided to 
include them in this section as BizTalk Server, since version BizTalk 2013 R2 can receive. 
and send messages using ISON pipelines. 

JSON isa lightweight data format, natural for humans to read and write. I's widely 
used in devices communication through modern API interfaces and client-side Java 
scripts 

"The main benefit of the JSON format and the reason why it became popular very 
fastis the reduced data size. In XML messages, there are lots of repeating element and 
attributes names, 
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Figure 1-11 shows an example ofa JSON message. 





ï 
B 


Figure 1-11. Example ofa JSON message 
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Envelope Schemas 


‘You can use envelopes to wrap one or more XML instance documents into a single XML. 
instance message. You could usually find them in a typical batch file that contains one or 
more child schemas. 


Property Schemas 


BizTalk Server uses property schemas for property promotion. Property promotion is the 
process of extracting feld element values from a message and inserting them into the 
message contest. We will see this topic later in the book. 

Now that you have learned how BizTalk Server uses the XML technology, itis time to 


reveal what is happening under the hood of the black bos. 





The Message Engine 


‘What is it that makes BizTalk Serv 
Server engine, it is essential to understand the different parts that help the engine 
provide the functionality of receiving, processing, and sending information, BizTalk 
Server bases the engine in the publish and subscribe model (see Figure 1-12), where 
incoming information is published to a central store and, in later stages, consumed by 
subscribers. There are many publish and subscribe models, but BizTalk Server works 
only ing the content of the publication itself. See Table 1-1. 


work? Before getting into the details of the BizTalk 
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Figure 1-12. The publish and subscribe architecture. 


Table 1-1. Relationship Between the Standard Publish and. 
Subscribe Model and BizTalk Server Elements 





Publish and Subscribe Model BizTalk Server Concept 





Pubicatons Messages 
Publishers Receive ports and orchestrations 
Subscribers Send ports and orchestrations 
Store Message Box database 





In this model, subscribers sign up for the types of publications that are of interest 
to them by setting up several requirements that publications (messages) must meet. 
The message is evaluated at the moment of publication, and all subscribers receive 
a copy ofi. Itis crucial to note that BizTalk Server does not use the content of the 
message only, but also uses additional information provided by the context of the 
message. See Figure 1-13, 
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Publishers — | [Publishers / Subscribers Subscribers 








Publications 
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=| — 


Figure 1-13. BizTalk Server implementation of the publish and subscribe architecture 






































In the next sections, you learn about these elements and how they relate to each other 


Host and Host Instances 


you have a developer background, it might help to extrapolate the host and host 
instance definitions with classes and object instances. In the object programming model, 
developers create classes (with methods and properties) that define how the objects of 
that class will behave when they are instantiated as object instances. You can think of 
the same way for host and host instances. The host defines how an instance of that host. 
will behave regarding the execution model (32-bit or 64-bit), tracking functionality, and 
‘much more. Once a host instance is created using the BizTalk administration console 

‘or any other supported method such as WMI, PowerShell, or the OEMExplorer DLL, 
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it inherits the host configuration to implement all the desired functionality and the 
correspondent windows service is created. Therefore, in other words, a host instance 
be defined as the physical implementation of the host definition (through a windows 
service) 

The host instance is responsible for initiating most of the message engine 





functionalities, and because it is a Windows service, it will do that by using its own set 
of hardware resources. This is a very important topic to understand because if there is 

a bottleneck related to hardware resources consumption, BizTalk Server functionality 
will not work, or it will work but ata very slow pace. In Chapter 4, you learn more about 
hostand host instances and how to tune them to extract the maximum value out of your 
hardware resources and configuration. 


Publications: The BizTalk Server Message 


Publications are the central exchange element in all publish and subscribe architectures, 
and BizTalk Server implements them by using the BizTalk message. 


What Is a BizTalk Server Message? 


Internally, 
content ofthe message, the information sent by the source system, is called the body 
part of the message (see Figure 1-14), 





rou can consider every BizTalk message as a multi-part message. The actual 





^" 





Ezzu erue ec ecu 
Eu 





— —— 





Figure 1-14. Message body part 
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Note Most BizTalk Server adapters create messages with only one part. However, 
the POPS adapter can create several parts to construct the message definition. 





Messages are created at the receiving stage using streaming me 





nisms provided 
by the receiving adapter and pipelines and implementing the following interfaces: 


Microsoft. BizTalk. Message. Interop. IBaseessage 
Microsoft. BizTalk.Nessage. Interop.IBasePart interfaces 


Since atthe: 





'oment of publishing messages to the database you could potentially 
have more than one subscriber, the engine might not know the number of active 
subscribers that are expecting that message. 





Note As all subscribers must receive a copy of the original untouched message, 
messages are immutable. This means that as soon as a message gets published, it 
cannot be modified using any method. 





Atthis point, ifyou are an experienced BizTalk Server developer, you might think that it 
is possible to modify messages in an orchestration by calling an external NET component. 
‘True, Howeve 





& to accomplish this ask, BizTalk Server forces you to insert a construct 
shape in the orchestration and this action indeed publishes a new message into the 
Message Box database. This is when orchestrations can also take the publisher role, 

BizTalk Server uses the Message Box database o store all messages using the table 
structure shown in Figure 1-15. 





Figure 1-15. BizTalk Messages structure. 


Notice that the uidiessagelD is present in all tables. When a message gets published, 
the engine assigns a unique MessageID property by generating a new GUID. 
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‘The Spool table is probably the most important table in the Message Box database. 
1 contains references to all the published messages. We will see this table in more detail 
[ater in the book, asit is related to several important areas. 

All BizTalk messages are multi-part messages; this means that they are composed of 
zero ton parts. The MessageParts table is used by the engine to store them. 

Every partis composed of fragments. As soon as a message gets published, BizTalk 
Server evaluates the size of the binary data and inserts several chunk fragments into 
the fragment tables. The large message size setting regulates this process. You will earn 
about this setting in Chapter 4, 


The Message Context 


When a documentis received, an important BizTalk Server artifact called adapter 
attaches context data to the document. This data is called the message context, as shown, 
in Figure 1-16. 





Receive Port 





Context Properties 





Figure 1-16. Message context creation 
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The message context isa container for several extended properties that BizTalk 
Server uses internally to operate and for routing reasons, Each property in the message 
contest has four elements 





+ Name: The actual name ofthe property. 
+ Value: Fulfilled by the adapter or the pipeline, 


‘+ Type: It could be Promoted or Not Promoted (detailed later in this 
chapter). 


+ Namespace: When adapters fulfil properties, they assign them to a 
specifie namespace. For an in-depth dive into BizTalk namespaces, 
see the previous section ofthis book. 


We can classify properties nto the following categories 
+ System-telated 

+ Distinguished properties 

^o Usertelated 

. Adapter-related 


Let's review all of them. 


System-Related Properties 
BizTalk uses these properties to populate valuable information about system properties like: 
«o ReceivePorthane: Name ofthe port that received the message. 


^ InboundTransportType: Name of the adapter used to receive the 
message. 

+ ReceiveLocationNane: Name ofthe location used to receive the 
message through the port. 

1+ MessageType: Probably the most important property of the BizTalk 
engine. We discuss it in the following section. 


Out-of-the-box adapters and the BizTalk engine promote properties into the context 
to implement BizTalk Server functionality. All system promoted properties are created 
using any of the following namespaces: 
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+ hitp://schenas.microsoft. con/BizTalk/2003 /nessageagent properties 


+ hitp://schenas.microsoft. con/BizTalk/2003/nessagetracking- 
properties 


. hitp://schesas.microsoft. con/BizTalk/2003/systen-properties 
There are tons ofsystem-related properties and documenting all of hem is out of he. 

scope ofthis book. 

Distinguished Properties 

A distinguished properties are written into the message context using the following 

namespace: 

http: //schenas. microsoft. con/BizTalk/ 2003 /btsDistinguishedFields. 


Developers can create custom distinguished properties (see Figure 1-17) to access 
specific business data through orchestrations. BizTalk Server can create them at the 
‘adapter or pipeline level using Microsoft Adapters (Swift isan excellent example ofthis). 














Figure 1-17. Distinguished properties 
User-Related Properti 


Developers can create custom context properties in two ways 





+ Creating a custom adapter that creates custom properties. 
+ Creating a custom pipeline that adds or changes properties to the context 


All user-related properties use custom namespaces, and they usually appear 
associated with the target namespace property of the incoming XML. 
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Adapter-Related Propertie 


"The adapter uses these properties to fulfil information related only to the adapter. 
Examples ofthese properties are: 





. FileCrestionTine:Creation time of the actual fle. 
«o ReceivedFileNane: Full path and name ofthe file. 
Alladapter-elated properties are created under the namespace: 
http: //schenas microsoft. con/BizTalk/2003 /adapterhase-properties. 


Where adapterNane changes based on the adapter used, 
Fora detailed list ofall context properties generated by BizTalk engine and its 
adapters, visit https ://nsdn.microsoft. con/en-us/ibrary/23562116. aspx. 


The Message Type Property 


As mentioned previously, when BizTalk Server receives a document, it generates all 
the context properties for the message. Among all of them, the pipeline construets the 
WessageType property. It is composed of the target namespace plus the main root node 
name, as shown in Figure 1-18, 


















Figure 1-18, MessageType property view 
In Figure 1-18, the received message has the following target namespace propery 
http: //FlatFLlekizard. BookOrdersConpleted 
And the following XML Main Root node name: 


BookOrdersConpleted 


a 


CHAPTER! REVEALING THE BLACK BOX 


Pipelines, within the disassembling stage, concatenate both values using the hash. 
character, #. Asa result, the following MessageType property is generated: 


itt 





JFlatFileiizard.BookOrdersConpletedfBookOrdersConpleted 
Notice two important things 


+ The property type is promoted. This process will not only create and 





insert this property into the context, but it will also promote it so that 
it becomes accessible for routing and internal engine operations. 
The XML and fla file disassemblers promote the message type on. 
the light, as they are processing the message. Ifyou are developing a 
custom pipeline component, you should implement a disassembler 
stage that promotes this property (ensuring proper routing). 


+ The namespace used is http://schenas microsoft .con/ 


BizTalk/2003/syten-properties. This namespace indicates that the 
system has generated the proper 








Note Message type property is a crucial concept in BizTalk Server, as the engine 
and developers use this property to identify a received message, for routing 
proposes, advanced developer tasks, and even for troubleshooting issues. 








Since the pipeline generates the message type property, what is happening when you 
set up the receive location to use a pass-through pipeline? Well, in this case, the message 
type property is not generated as no disassembling stage could be executed. This is the 

typical scenario where BizTalk Server is receiving a binary chunk of data, and the logic 
‘of accessing the content of the message is implemented using custom code (by calling a 
NET component or an expression shape in orchestrations) 





Message Properties 


Message properties are the properties generated by the adapter and pipelines that 
belong to the message context. As outlined in previous sections, properties can be 
‘written to the context or promoted into the context. The message property type reflects 
this concept. The type could be Promoted or Not Promoted, as shown in Figure 1-18, 
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Figure 1-19. Types of message properties 





Note The difference between these two types is that promoted properties can be 
used as conditions in message routing, while written properties cannot. 





Non-promoted properties can be distinguished, allowing orchestrations to access 
them using expression shapes. As you have learned previously, these properties belong. 
to the following namespace: 





http: //schenas microsoft. con/BizTalk/2003/btsDistinguisedFields. 


‘The adapter ean promote properties directly from written properties. However If you 
want to add custom promoted properties, you can do so in two ways 





+ Based ona schema definition. This is the situation when you want 
to route based on an element in a specific message. In this case, 
the promoted property is assigned to the Microsoft XLANGS 
BaseTypes.MessageDataPropertyBase type. 


+ Any oth 
situation when 





property not based on a schema definition, This is the 
Jou want to create a custom promoted property 
inside a custom pipeline component and access it later, In this 
case, the promoted property is based on the Microsoft .XLANGs . 
BaseTypes.MessageContextPropertyBase type. 





Both procedures require that you a ereate a property schema that contains a 
definition for the property. Also, you must deploy the property schema to BizTalk Server. 
‘management database. 
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Distinguished Properties (fields) 


As mentioned in the previous section, schema elements can be distinguished tobe 
accessible by orchestrations during design time and using expression shapes. Promoted 
properties and distinguished properties are both writen into the context, and that is why the 
‘concept can be deceiving. However, you could easily understand the difference between the 





wo of them because distinguished properties diez from promoted properties in twa way 
+The property type is Not Promoted. 
‘+ They do not require the property schema file. 


For these reasons, you cannot use distinguished properties for routing proposes as 
the engine cannot access them at the moment of evaluating the subscription. 


Message Properties Design Considerations 


‘You should take the following points into consideration while thinking of promoting and 
‘writing properties into the message context: 
+The maximum length of promoted property is 256 characters, 
‘This limitation exists to guarantee smooth performance. In the 
POP3 adapter the part name could be created with a very long 
name description. This will cause engine errors because message 





properties can reference the part name. 
Something similar can happen for the fil 
length of rhe fle path, file mask, and B 
characters. Keep this limitation in mind, 





idapter as the total 
name cannot exceed 256 





+ Record nodes cannot be distinguished, 


+ Record nodes can be promoted if they are non-repeated records 
(single nodes). 


Non-promoted properties have no length limitation, as you cannot use them in 
context routing. Examples of written properties are: 


+ Distinguished fields 


+ Non-promoted system properties written to the context 





‘+ Custom properties added by developers through a custom pipeline 
component or custom „NET adapters 
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Message Properties Performance Recommendations 


"The following is a list of performance recommendations that you should consider at 
carly stages of project development. Take all of them very seriously, especially ifyou are. 
developing a low latency solution 


Reduce the number of written and promoted properties and 
eliminate those that are not strictly necessat 





XPath expressions can be very long, primarily when the element is 
located very deep in the message. Therefore, the more distinguished 
fields you have, the larger the context size. This situation affects the 
overall performance of the platform. Whenever possible, consider 
moving the deep elements atthe beginning ofthe schema. 





Tuis recommended to reduce the property name length as much 
as possible, Shorter names ensure that the engine consumes 
less memory and still provides business functionality. This fact 
is especially true for distinguished fields as they do not have 
256-character limitation. 


you are not planning to use the property for routing, do not 
promote it! Just distinguish it. Promoted properties consume more 
resources asthe engine inserts them into the subscriptions table, 
while distinguished properties are not inserted. Also, if you enable 
property tracking, SQL Server process consumes more memory, 
processor, and IO resources as it must insert the tracking information 
into the tracking data tables within the message box and then to the 
tracking database. These two facts might not be relevant while you 
are coding, but when the solution goes ive and must process millions 
ofinstances per day, believe me: it matters! 





Especially for flat file scenarios, performance is affected by the 
position ofthe promoted property within the schema definition 
Promoted properties are found faster if you position them at the 
beginning ofthe schema. 
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+ Ifthe messages are small (fewer than 100 kilobytes), you can de- 
serialize the message into a NET class object and access the publie 
static fields and properties (Instead of using XPath). If the message 

ds complex business rules, accessing data using the properties 
exposed by an Instance ofa NET object is faster than using XPath, 





expressions because XPath loads the full message into the memory 
every time it executes, 


The following code shows an example ofa serialized message that exposes 
distinguished properties: 


using Systen; 
using Microsoft. XLANGs. BaseTypes; 


nanespace NetClass 
( 
[Serializable] 
public class MyBookNanespace 
( 
public MyBook() 
t 
ASN = "101928818910111"; 
bookTitle = "BizTalk Server 2016 book"; 
) 


[DistinguishedFieldattribute()] 
public String iSBN; 


[DistinguishedFieldattribute()] 
public int bookTitle; 





ned within 
the context ofa construct shape because the properties are implementing the 
DistinguishedFieldAttribute (defined in Microsoft. XLANGs .BaseTypes). The following 
code is checking ifthe ISBN property equals "101928818910111" and i yes, it changes the 
bookTitle property to "BizTalk 2016, performance tuning and optinization" 


The code to create an instance of this object should be impl 
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msghessageIn = new MyBookNaneSpace.MyBook(); 
if (msgħessageIn.iSBN== "1019288:8910:11") 





t 
neglessageln.bookTitle -"BizTalk 2016, performance tuning and 
optinization” 

i 

Subscriptions 


In the publish and subscribe architecture, subscriptions are the link between the 
publication and subscribers. For BizTalk Server, a subscription isa set of conditions 
statements called predicates. The BizTalk Server engine creates these statements based on: 





+ Values extracted from the message context 
^o Values related to the subscription itself 
Predicates are saved in the Predicates tables within the Message Box database, 
while subscription-related information like priority, order delivery sequence, and 


convoy configuration is stored in a SQL Server table called Subscriptions. The action of 
saving data to these tables is executed when the subscriber (normally ase 





portor an 
orchestration) is enlisted. At this stage, the following two stored procedures are called by 
the engine: 
. Bts_CreateSubscription HostNane—Inserts data into the 
Subscriptions table. 
+ Bts InsertPredicate HostNane- Inserts data Into the Predicates 
table. 

When you deploy BizTalk artifacts, like ports or orchestrations, and you do not enlist 
them, the message engine does not insert the subscription information into the previous 
tables. This means that the subscription is not active and because of that, the engine 
will not forward messages to any subscriber (creating a routing failure report —RER— 
attached to the message context), 

"The following predicates tables can be found in Message Box database: 

+ LessThanPredicates 
+ GreaterThanPredicates 
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+ NotEqualsPredicates 
+ LessThanOrEqualsPredicates 
+ GreaterthanOrEqualsPredicates 
+ FirstPassPredicates 
+ ExistsPredicates 
^o EqualsPredicates2ndPass 
^o BitiseADPredicates 
^o EqualsPredicates 


Using the Information stored in this set of tables, the Message Agent and the End 
Point Manager decide how to route messages to subscribers, 

For Instance, iL you have a send port called sndPortSubscriptions and you create a 
send port filter with the following expression: 


BIS.ReceivePortNane = rcvPortSubscriptions 


Atter enlisting the send port, BizTalk Server inserts the information shown in 
Figure 1-20 into the Subscription and EqualsPredicates tables (see Figure 1-21). 


—— - 
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Figure 1-20. Subscription table information 
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Figure 1-21 shows the EqualsPredicates table. 


[23 — 
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Figure 1-21. EqualsPredicates table 


‘The engine inserts the condition statement into the EqualsPredicates table because 
you used the equals operator to build the fter expression (=). 
BizTalk Server works with two types of subscriptions: 


1+ Activation Subscriptions 


+ Instance Subscriptions 


Activation Subscriptions 


These types of subscriptions activate a new instance of he subscriber when a specifie 
message is recelved, For instance, in the following cases you can find activation, 
subscriptions 


^ Send ports with filters 
+ Send ports bound to an orchestration 


+ Orchestration receive shapes with the Activate property set to true 


‘Send Ports with Filters 


‘The message engine evaluates subscriptions, and when a send port filtering expression. 
is met, the send port is activated and the message is sent out, See Figure 1-22. 
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Figure 1-22. Send port filter exampleSend ports bound to an orchestration 


‘This situation is very similar to the previous scenario, as the message engine 
‘works in the same way, but in addition, the orchestration ID is part of the condition 


that activates the send port, and itis automatically inserted into the EqualsPredicate 
table, See Figure 1-23. 
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Figure 1-23. Send port bound to an orchestration 
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Orchestration Receive Shapes with Activate Property Set to True 


In this case, the BizTalk engine will start an orchestration instance when the receiving 
ports recelvea new message, Additionally, you can add filter expressions to the 
subscription, as shown in Figure 1-24, 























— 
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Figure 1-24, Receive shape with activation and filter subscriptions 


"The previous orchestration is extending the subscription by picking up messages that 
‘contain an error report structure within the context. 
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Instance Subscriptions 


This type of subscription does not activate à new subscriber. Messages received 
are routed ro existing orchestrations instances that are already running or waiting 
fora response using a Request/Response (receive port) scenario or an interchange 
regulated by a correlation. Ifthe corresponding subscribers are not running anymore, 





subscriptions are automatically cleared. 


Request-Response Scenario 


In the example shown in Figure 1-25, BizTalk Server activates the request-response port 
ration. 








nds the message back to the caller (response operation). 





p . 


























Figure 1-25. Request-response port with the send activation 


Solicit-Response Scenario 


In this case, BizTalk Server activates the orchestration when the solicit-response port 
receives a new correlated message. See Figure 1-26 
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Figure 1-28. Solicit-response port with correlation 


Subscription Priority 


When the engine generates a subscription, the creation process sets a priority level for 
that subseription into the Subscriptions table, as shown in Figure 1-27, 
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Figure 1-27. Priority of subscriptions 
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Priority values range from 1 to 10, 1 being the highest priority. Orchestration-related 
subscriptions have a value of 7. While changing this value directly in the table is not 
supported, it might be very interesting to test the behavior of orchestrations with a 
different value, 

At moment of writing this book, the only situation where users ean change priority 
levels is atthe send port level. Physical send ports have an advanced transport section. 
‘where you ean set up priority, as shown in Figure 1-28. 
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Figure 1-28. Priority of send ports 


"This setting is very useful when you want BizTalk Server to send messages to a 
specifie destination system with a higher priority. For Instance, iL you change the 
Send Port priority to 1, this action is reflected in the Subscriptions table, as shown. 
in Figure 1-29, 
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Figure 1-29. Priority at Send port level changed to 1 


From now on, every time BizTalk Server has to send messages using this port, it will 
assign the highest priority as the stored procedures that query the host queue tables will 
retrieve these operations first 


Searching for Subscription Information 


When troubleshooting, you might need to search for a specifie subscription. You can use 
the BizTalk Administration Console to query subscription information. To do that, follow 
these steps: 

1. Open the BizTalk Administration Console. 


2. Click on BizTalk Group and Select the New Query tab, as shown in 
Figure 1-30. 
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Figure 1-30. Accessing the New Query tab 


CHAPTER! REVEALING THE BLACK BOX 


3, Inthe Search for option, choose Subscriptions, as shown in 
Figure 1-31. 














Figure 1-31. Search for option 


4, The query will list all subscriptions in the system, as shown in 
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Figure 1-32. Visualization of the Subscription type 


Notice the Subscription Type column. I shows ifthe subscription is an Activation 
or an Instance subscription, Let's change the query now to filter Instance Subscriptions 
only. 
5, Add the Filter Subscription Type to the query, as shown in 
Figure 1-33. 
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Figure 1-33. Subscription Type filter 


6. Select Instance Subscription as the subscription type, as shown in 
Figure 1-34 








Figure 1-34. Subscription Type filter (Instance Subscription) 


7. Click Run Query. All instance subscriptions will be shown, as you 
can see in Figure 1-35, 














Figure 1-35. Instance Subscription results 


3. You can now right-click any of the query results to access the 
Subscription Details dialog bos, as shown in Figure 1-36. 
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Figure 1-36. Subscription Details dialog box 





Note You can also obtain the subscription information using the BHM tool and 
accessing the Subscriptions query. 





Publishers 
Publishers are BizTalk Server artifacts that publish messages to the Message Box 
database. BizTalk Server works with the following publisher types: 

+ Receive ports: Usually, the main entry point for BizTalk messages. 


*— Orchestrations: Every time an orchestration creates a new 
message using a construct shape, it publishes a new message to 
the Message Box. 








Note Keep in mind that construct shapes are used in combination with message 
assignment or transform shapes only 
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"The publication process (Figure 1-37) is initiated mainly by the host instance process 
that is running receive locations and/or orchestrations. It executes a set of SQL Server 
stored procedures that evaluate the subscriptions and insert the message inthe right 
host queue table. This process is outlined in Figure 1-37. 
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Figure 1-37. Message publication process 
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Here are the detailed steps: 


1. The message agent component that is running in the host instance 
Inserts the values ofthe promoted properties and predicates 
values from the message context into the Message Box database, 

doing this, the message agent can take routing decisions in the 

dequeue process explained later, 





2, The BizTalk host instance process calls the stared procedure 
dnt evaluate subscription.Thisstored procedure queries 
subscriptions and predicate tables to verify subscriptions. 


3, Ma subscription with active subscribers is found, in other words, if 
there are orchestrations or send ports enlisted and expecting that 
specific message, the engine adds the message to the main queue 
table ofthat host and calls the Insert. Spool stored procedure to 
insert the message into the Spool table. 


On the other hand, if no active subscribers are found, the message 
is suspended and stored in the suspended queue of the same host, 


4. Then, the insert parts stored procedure inserts the binary data 
into the parts, messageparts, and fragments tables. 


5. Additionally, if message body tracking is enabled, the message 
is tracked locally within the Message Box database tracking data 
tables, and in later stages, a SQL Server job moves this data to 
the trackingOTADB, We review the SQL Server infrastructure in. 
Chapter, 


Subscribers 


In this section, you learn about all types of subscribers and how the message engine 
retrieves messages from the Message Box database and routes them to subscribers, 

As detailed previously, subscribers are BizTalk Server elements that process 
publications (messages) based an a set of conditions called subscriptions. BizTalk Server 
implements the following subscribers: 


+ Orchestrations. Orchestrations can be subscribed to messages using 
the following approaches: 
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^o Receiving messages from a Receive shape. 

+ Receiving messages directly from the Message Box database. 

+ Receiving messages from nested orchestrations. 


+ Send ports. Send ports can be subscribed to messages in the 
following ways: 


+ When they are bound to an orchestration 
+ When they are not bound to orchestration, but they have filtering 
conditions (message routing). 
^o System. The message engine creates a special internal subseription in 
these situations: 
+ Correlation scenarios. 
+ Calling an inline pipeline inside an orchestration, 


Subscribers receive their a copy of the message through a process called dequeue. 


The Dequeue Process 


BizTalk host instances execute a stored procedure called bts DeQueueessages within 
the Message Box database. The frequency at which these calls occur is regulated 
by a host setting called the Messaging Polling Interval and by default is set to 500 
milliseconds (twice per second). 

As messages are published, host instances poll their respective queues within the 
Message Box database to retrieve new messages. 





Note While there are rows in the host queue table, the Pooling Interval setting is 
not used, but once the queue has zero rows, the host instance will poll the queue 
at intervals based on this setting until more messages are published. 





‘You can find this setting using the BizTalk Administration Console by clicking on 
settings action, as shown in Figure 1-38. 
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Figure 1-38. Pooling intervals for BizTalk Server hosts 





Note There are two settings: 


Messaging Frequency at which host instances poll the Message Box database to 
retrieve new messages to subscribers, 


Orchestrations Frequency at which host instances poll the Message Box database 
to run new Orchestration instances. 

By default, BizTalk Server is configured to perform under balanced latency 
‘conditions (closer to high throughput scenarios rather than low latency). 
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"The best practice is to leave the default balanced setting of 500 milliseconds unless 
the host has specifie low latency requirements, In these circumstances, lowering this 
value can reduce overall latency but, on the other hand, SQL Server and BizTalk Servers. 
can consume more resources as the host instances perform more round trips to the 
Message Box database. 

You will see how to adjust this setting for several scenarios inthe book. 











Warning! Decreasing the value of pooling intervals may cause excessive CPU 
utilization on the SQL Server computer that houses the Message Box database 
Instance. 





When the bts DeQueuetessages stored procedure is called, it analyzes the 
Subscriptions, Instances, and HostMainQueue tables and loads batches of 20 messages 
into the in-memory queue of the host instance. The EndPoint Manager Subservice sends 
the messages to the original subseribers, designated by the Subscription table, and 
subscribers deliver the appropriate message to the destination systems. 

Once all the active subscribers use that message, the engine marks that message 
as consumed, and it removes it from all the Message Box tables (calling a set of stored 
procedures and SQL Server agent jobs) 

Figure 1-39 outlines the fll process 
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Figure 1-39. BizTalk Server Dequeue process 


Adapters 


‘The adapter is used to exchange messages with source and destination systems. Most 
ofthe adapters are NET components that have the protocol knowledge to access and 
transfer messages through an NTFS folder, a WCF service, a SQL Server database, and 
‘many others, By design, BizTalk implements receiving and sending adapters. On top 
ofthe apparent action of receiving messages, the receive adapter also adds properties 
Io the message context, Some of them are generic and exist in all adapters such as. 
ReceiveLocationNane, receivingPortnane, nessagelD, and processingServer. Others 
are related toa specific adapter. 

‘As outlined in previous sections, the receiving adapter also sends the message to the 
engine to check for active subscriptions, 
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‘When all host instances start, all the receive adapters that have receiving locations 
enabled are instantiated. However, send adapters workin an entirely different way: All 
‘out-of-the-box BizTalk send adapters are not instantiated when host instances start; they 
do this ina way that is called lazy creation. This means that the adapter starts when the 
Message engine gets the first message from the internal message queue and delivers it 

to the send port. By acting this way, BizTalk Server saves system resources when send 
adapters are not configured in send ports. 

In some scenarios, BizTalk applications require Interaction with systems that cannot. 
be accessed using the out-of-the-box adapters, For this reason, third-party companies 
‘have written adapters to support additional protocols. After you review all the available 
adapters, you might reach a point in which you need to develop an adapter, Writing a 
‘custom adapter is one of the most challenging things in BizTalk Server. The Adapter 
Framework, in combination with the examples in the SDK, is meant to simplify the 
creation process. This is out of the scope of ths book; therefore, if you have a specific 
‘need, you can search in MSDN, and you will find plenty of public resources related to 
this task. 

As BizTalk uses the publish and subscribe architecture, the engine needs to store 
messages and subscription information. BizTalk Servers uses a SQL Server database 
called the Message Box database. 











The MessageBox Database 


While all Bi Talk Server databases are essential, the Message Box database is the heart of 
the engine and gives BizTalk Server the ability to store the following information: 


‘+ Subscriptions which were detailed previously in the Subscriptions 
section. 


. Binary message data 
. Host queues 
«o Spoot table 
+ Orchestration persistent points 


+ Debugging information 





* Tracking temporary data 
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Host Tables 


Host tables area set of tables within the Message Box database that are created as part 
ofthe process of adding a new host to a BizTalk Server group. These tables are used in. 
message publication, the dequeue process, orchestration states, and host throttling 

mechanisms, They are classified into two categories: Queue tables and System tables, 


Queue Tables 


"These tables are used by the 





ngine to access messages that are related to a specific host. 
+ Main quewe—Contains references to messages that are pending to 
deliver to a host and are not suspended. Ideally, this table should not 
grow too large (the threshold for raising an alarm is directly inked to 
performance service level agre 
table indicates the number of messages waiting ro be processed fora. 
host 





ements). The number of rows in this 


+ Suspended queue Where references to suspended messages are 
stored This is important, When a message gets suspended It remains 
in the Message Box until resume or terminate actions occur So ifthe 
suspended queue is growing, performance is affected. A suspended 
message can occur due to validating errors, failed transmissions, or 





the impossibility to find an active subscription (among others). 


^o Scheduled queue— Contains documents that have been processed 
and are waiting to be sent based on the service window that was 
specified for the port. 


+ InstanceStateMessageReferences—The State queue table saves the 
list of messages that have been processed by an instance but might 
be needed later. This is used main 





in orchestrations when the 
developer creates a BizTalk message using a construction shape. 
Because the message might he used everywhere In the code, the 
Message engine saves a reference to itin the State Queue table, 
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System Tables 


These tables are used 





y the engine to create temporary information related to several 
engine functionalities, like subscription generation, dequeue processing, and routing, 


^ MessageRefCountLog—When a message has more than one 
subscriber, the BizTalk Server engine use this table to correlate this 
information with the rest of reference tables. Every time a message 
isused, the engine increments a reference counter within this table. 
When this counter decreases to zero, the message is marked as "to be 
purged” and SQL Server jobs will remove references to that message 
so that subscribers will not pick them up again. 


+ DynanicStateInfo- Stores all orchestration persistent points. Itis 
saving real-time data, so if you query this table without having any 
running orchestration, it will be empty. Dehydration and rehydration 
processes will use this table to restore orchestrations to previous. 


+ DeQueueBatches—When a message is delivered to all the available 
subscribers and it is not in the InstanceStateMessageReferences 
table, it is inserted into this table, so the bts_DeQueve stored 
procedure can delete all references to the message. 


Spool Table 


‘The Spool table is an important BizTalk Server table because the engine uses it to hold all 
ferences (binary data that's saved in the fragment tables) ro messages that 
are sil alive in the system (see Figure 1-40). Alive messages are: 








^ Messages that are being consumed in active service instances 
(Subscribers to that message are orchestrations and send ports 
mostly). 


^ Messages that are queued because they need to be sent toa 
scheduled send port. 


^ Messages that are being used by retrying send ports. 
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Figure 1-40. Messages table structure and spool table relationship 





Knowing the Number of Messages in the Spool Table 


"There are two ways to gather the number of rows in the Spool table: by counting the 
number of rows of the Spool table or by querying the Spool performance counter. 
‘To count the number of rows of the Spool table, use the following: 


SELECT COUNT(*) FROM Spool WITH (noLock) 





"The use ofthe WITH (NoLock) statement here prevents the query from affecting 
locked data. Itis essential to keep this in mind when performing SQL Server queries 
against a BizTalk Server database. This is especially true for the Message Box database 

"To query the Spool performance counter, choose BizTalk > Message Box > General 
Count 





This counter tracks the size of the spool over a Message Box database on a 
specific server Ifthe environment is using multiple Message Boxes, you should query 
the Message Bos that is acting as a publisher. 


Spool Table as a Performance Ini 





ator 


BizTalk Server can queue messages for several reasons. For instance, imagine that 
destination systems are not available for an extended period. In this scenario, BizTalk 
Server cannot send messages through those send ports, What would happen with the 
Spool table in this scenario? 

As you might have guessed, the Spool table will become larger since BizTalk Server is 
‘keeping those messages as suspended, and it will not remove them until the user decides 





‘what to do with them. In this case, you would typically also see that the average total 
documents received per second performance counter is greater than the documents 
processed per second performance counter (because BizTalk cannot dequeue messages. 
to the subscribers) 
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In this situation, the system might have a critical performance issue because BizTalk 
Server cannot deal with the current load. n other words, the load is not considered 
sustainable. This contest can eventually induce a throttling condition (based on. 
database size) that causes BizTalk Server to perform slowly as message publication 
is begin throttled, This is especially true if racking is enabled, as the calculation for 
message count in the database considers the number of messages in the Tracking Spool 
tables within the Message Box database 

‘As you can see in the example in Figure 1-41, the Spool table could be used tod 
seactive problems and to monitor the platform for increasing trends over an extended 
period. We cover these analysis techniques in Chapter 2. 
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Figure 1-41, Messages queued in the Spool table 





Summary 


In this chapter, you learned how BizTalk Server uses different XML technology elements 
such as namespaces, attributes, and elements to create a solid foundation for message 
definitions. You also learned about certain engine behaviors that ensure the functionality 
of the product. 

Also, the chapter went through most of the relevant topics to clarify how BizTalk 
Server processes information using key elements of the engine, such as messages, 
publishers, subseribers, and the Message Box database, and how it integrates all of 
them by implementing the publish and subseribe architecture, Although most ofthis 
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‘functionality is implemented by the publishing and dequeuing processes, there are 
several aspects like subseription priority large message size settings, and host and host 
instances configurations that have a direct impact on how BizTalk Server integrates all 
the pieces together. 

Now that you have a deeper picture of what is happening under the hood of BizTalk 
Server, is time to gain solid knowledge about performance analysis. In the next chapter, 
‘you see how you can interpret performance information, a subject that is crucial to learn. 
how to assess your production environment and to interpret the output of your testing 
procedures 

‘And remember, do not ever say again: "What is happening with this black box?” 
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Performance Analysis 


Analyzing performance data is not an easy task. There are dozens of performance 
ind you do not know how to analyze the information correctly, that ean 
mislead you in so many ways and eventually it will cause you get th 
Additionally, hardware and features are rapidly 
and analysis methodologies may need to evolve as well. However, as performance 

s are always based on samples over time, you can apply the concepts detalled 
in this chapter to make your life easier when trying to find an issue or a bottleneck. In 








wrong conclusion, 





solving, and your performance testing 





this chapter, you will learn general analysis techniques that you can use to troubleshoot 
performance issues. Although the examples used focus on BizTalk Server, you could. 
extrapolate them to any other software that exposes thelr performance counters 





In this chapter, you learn about the following topics: 
+ Performance analysis techniques. 


+ Performance analysis guidelines in terms of the most used threshold 
for specific counters 


^ How to interpret the latency counters 
+ BizTalk Server throttling 
+ Suspended messages 


+ Tracking 


Performance Analysis Techniques 


In this section, you learn to use the most common performance analysis techniques that 
will help you understand performance data. 
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Increasing Trends Over Long Periods 


Growing trends can often indicate resource leaks or data contention, This is especially 
true for BizTalk Server counters that are related to queue information such as Spool, 
number of instances, host queue counters, suspended messages, and all BizTalk Server 
latency counters. On the contrary, some cumulative counters such as VTCP Connection 
Failures will increment indefinitely whenever failures occur, and the underlying problem 
‘could not be related ta data contention or resource leak. 

‘To analyze these kinds of situations, you can trace an imaginary line that links all the 
lower values. By doing so, you are detecting not only ifa specific resource is leaking, but 
also ifthat the leaking condition is getting worse over time. 

the imaginary lines increase the angle of the previous line, the leaking condition 
gets worse over time. However, ifthe opposite occurs, it indicates that the situation is 
getting be 

To illustrate this scenario imagine that the Spool performance capture over a 
24-hour period, as shown in Figure 2-1 











rover me. 























Figure 2-1. Increasing trend over the Spool table. 
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As you can see, messages are getting queued in BizTalk as the Spool table is gr 
over an extended period. On top of that, the situation is getting worse, because th 


* 


arrows A, B, and C have a more significant upward trend. 





Note Some performance counters are cumulative and show increasing trends as 
they calculate values in the form of: 


Counter Value = Counter value + n 
The counter System Up Tine isa perfect example of a cumulative counter, 





Figure 2-2 shows server uptime in seconds. It does not indicate a resource leak. 
Some of the BizTalk Server related counters that exhibit leaks or data contention with 
Increasing trends are: 


+ Spool 
+ Suspended messages 
+ Hostqueue length 


* All the latency counters (inbound, outbound, and request-response) 




















Figure 2-2. Increasing trend over the Spool table 
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System counters: 
+ Thread count 
+ Working set 
+ Private bytes 
+ Decreasing tends over a long period 


+ Decreasing trends usually indicate resource exhaustion (see Figure 2-3) 























Figure 2-3. Decreasing trend over the Spool table 


Examples of counters that exhibit resource exhaustion with reducing patterns are: 
+ Available MBytes 
+ Free System Page Table entries 


Now, look at the following case forthe Spool perfor 
Figure 2-4. 





ice counter, shown in 
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Figure 2-4. Increasing trend over the Spool table 


What is happening here? The platform is keeping up very well with the current load 
unl the end of arrow A when something odd happens, The Spool counter shows à 
downward tend until the end of B. 

"Two things could be happening here: 


+ Theengine is processing messages faster than the rate of incoming 
messages. As the Spool table holds all the messages that are being 
processed at a given time, it could indicate that the engine is 
processing messages very efficiently and, because of that, no backlog 
is ereated in the Spool table. 





^ The platform is not receiving new messages and it is just sending 
whatever was pending in the Spool, If the platform is usually 
processing messages without interruptions, you should suspe 
all receive locations got disabled, or that IS is not working (if the 
platform is receiving most of the requests through IIS). 
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Crossing a Threshold 


Several performance counters indicate a possible bottleneck when the majority ofthe 
values cross a real or an imaginary line (a minimum value). Figure 2-5 shows the 
\BizTalk: Messaging Latency\Inbound Latency (sec) capture for a 24-hour period, All 
values are crossing the imaginary line created using 27 as the minimum value, 























Figure2-5, System Queue length crossing threshold. 


‘That means that the engine takes more than 27 seconds to publish a message. On 
later stages, you should investigate what resource could be causing the this issue. 





Inverse Relationships 


In most scenarios, analyzing performance data is all about finding the right cause and 
effect. Two or more performance counters often represent this inverse relationship. 
For instance, look at the example in Figure 2-6. 
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Figure 2-6. Inverse relationship with documents received and available memory 


‘The dotted 
and the blue line represents the available memory on the server. As you can see, while 
the platform is receiving messages, the amount of available memory also decreases until 
the graph reaches the red circle. That is the moment when the host stops receiving new 
messages (around 6:20 PM) and the available n 
Analyzing this kind of relationships is an excellent technique to find issues. 





low line shows the documents received by a specific host instance 











„mory increases to 1050 MB, 


Performance Counters Analysis Guidelines 


In this section, you learn how to analyze an essential performance counter set that will 
help you troubleshoot BizTalk Server performance issues more efficiently 

Keep in mind that this book provides you with a set of initial guide 
recommendations that for very specifie scenarios might not be suitable. T 


esand 








eldea isto 





provide you with a preliminary set of information that you can use as a starting point 
‘when it comes to BizTalk Server performance 
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Processor 


Itis a common mistake to analyze the processor time performance counter alone, 
you analyze it in isolation, without checking the rest of processor counters, it does 
mot provide proper conclusions or even can lead you to the wrong ones, 

When analyzing processor bottlenecks, you should consider the following 
performance counters: 


‘+ \Processor Information (*)\% Privileged Time. 
+ \Processor Information (")\% User Time. 
+ \Process (*)\ % Processor Time. 


+ \Processor Infor 





nation (*)\%e Processor time. 
‘+ \Processor Information (*)\Interrupts/sec. 


Another critical thing to keep in mind is that having a performance baseline of the 
server is key to concluding that your server s running with performance degradation 
As a general rule, any standard tasks that drive the processor time to 100%, for an 
‘extended period, should raise alarms. 

‘Once you have identified a processor bottleneck atthe BizTalk Server layer, you 
‘can use the following actions as an inspiration for compensating the lack of processor 
capacity: 


+ Add more servers (also known as scaling-out)— Adding another 
server provides double the resources (not only CPU), which will 
provide higher throughput 





+ Add more processors (also known as sealing-up)—Ifyou have 
detected that the cause of the processor bottleneck hides within any 
of the following factors, you should choose to scale-up instead of 
seale-out 





‘+ Large message transforms within the custom code, pipelines, 
or maps. 


+ Large number of messages for an interchange. 
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+ Divide the workload in the BizTalk Server farm-—Ifyour BizTalk 
Servers are running hosts that run all of the BizTalk Server. 
functionalities, you can optimize the resource consumption by 
dedicating the servers to run only one functionality ata time 
(receiving, processing, or sending). We will see host separation 
recommendations later in the book. 


+ Schedule tasks When the processor is idle or with low load, some 
operations such offline batch scenarios, backups, and any other. 
unattended processes that don't have to be completed immediately 
can be rescheduled for later processing. By doing this, you will 
be distributing the load during the day and that could alleviate 
the processor usage under high load. BizTalk Server 2016 offers 
advanced scheduling options at the part level, so you can enable this 
setting to distribute the load along the day (of course, if the business 
requirements allow you to do that) 





Percentage of Processor Privileged Time 


"This counter measures the percentage of time the processor runs in kernel mode, 
Applications like BizTalk Server, SQL Server, or Notepad run in user mode while 
‘operating system components run in kernel mode. Probably the essential differ 





between kernel and user mode is that, in user mode, threads run under the context. 

ofa process and because of that it has its own set of memory resources that are also 
protected and cannot be accessed from other processes. In kernel mode, all components 
share the same virtual address space. Therefore ifa driver running in kernel mode 





, the entire operating system can crash as well 





Examples of Windows components that run in kernel mode are: 
+ Filesystem 


+ Object manager 





+ Virtual memory managen 


As you can see, most of these components are related to hardware resource 


‘management. 
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For Instance, ifa BizTalk Server adapter access the network to stream data or ifa host 
Instance writes messages into the in-memory queue, these functions call components 
of the Windows operating system that runs in kernel mode. Imagine now that ina 
particular scenario, the BizTalk platform is receiving one million messages at the same 
time. Assuming that no BizTalk Server throttling conditions are raised, adapters will run 
kernel functions thousands of times a second to access memory, disks, and network 
resources. In this case, high privileged times can occur, and that indicates thatthe server 
is busy calling operating system componer 

The following able can be used as a general guideline to interpret the percentage of 
the privilege time performance counter. 





is to access those resources, 








Percentage Alert 





m Acceptable value. The server is not busy, 

>=20and<30 Waring. This situation alang with high user mode and high system queue. 
length or high context switching could be part f a processor bottleneck. 

>=30 Critical. Most likely the environment is suffering from a processor bottleneck 
during the analyzed period if performance degradation is perceived. 





Once you suspect you have a kernel mode issue, you can identify which process 
is the primary cause by analyzing the percentage of \Process (*) % privileged time 
(counter at process evel) 


Percentage of User Time 


"This counter measures the percentage of time the processor runs in user mode, As you. 
saw in the Privileged Performance Counter section, applications such as Microsoft 
BizTalk Server, Microsoft Word, and Microsoft SQL Server run in user mode. 

All processes (not related only to a core Windows component) execute a mix of user 
and kernel cade. 
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‘The following table can be used as a general guideline to interpret the percentage of 
user time performance counter, 





Percentage Aert 





E “Acceptable value. The server ls not busy 


Warning. This situation along with high privileged percentage could be part ot 
a processor bottleneck (especiali if the load increases) 





E] Critical. Most likely the environment is suffering from a processor bottleneck 
during the analyzed period (f performance degradation is perceived). 








Now that you also learned about the user time counter, in general, it is easter to 
diagnose user mode conditions than high privileged time. 

Why? Identifying the issue related to user mode is always more manageable as, at the 
end ofthe day, the responsible party is always a specific process. Moreover ifthe server 
is suffering from high privileged time, this situation will impact all running processes, 
especially while they are accessing I/O resources. 


Percentage of Processor Time 


Processor time shows the percentage of time where the serv 





is processing threads. As 
you learned previously, threads can run in user or kernel modes. Therefore, this counter 
‘sums the counters % User Time and Privileged Time. 

There isan unusual situation where hardware devices are falling and affect the 
ab 
‘what device Is causing the issue by analy 
ETW tracing for that propose, 





rall processor time by adding interrupt time. Ifa device in your server is working 





mally, it can increase the processor usage. Unfortunately, you cannot determine 





g performance counters, but you can use the 








Note Although this issue can occur, it is recommended that you analyze interrupt. 
time only when the value for this counter is higher than 10%. 
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The following table can be used as a general guideline to interpret the p 





tage of 
processor time counter, 





Percentage Alert 





<50 ‘Acceptable value. The processo is not causing an issue if system queue 
Jength is under optimum values. 

>=50and<=80 Warning. This situation along high context switching could be part o a 
processar bottleneck. t high context switching is observed, then reducing the 
number of active threads could alleviate the condition. 

>=80 “Critical. Most likey the environment is suffering from a processor bottleneck 
during the analyzed period if performance degradation is perceived. 





Based on this table, ifthe counter shows values higher than 50%, check the 
privileged and user time counters to investigate the cause, 

Keep in mind that high values for % Processor Time for the system process can 
indicate a busy device driver (like a network card). In this case, you could usually see 
high percentage of DPC time. DPCs are interrupts that run at a lower priority than 
normal interrupts (measured by % Interrupt Time). 





System Processor Queue Length 


"This perform 
but they are idle because processors are busy. However, it is not reliable especially 





se counter shows the number of threads that are ready for execution, 





virtualized sen 





When the server is physical, you can use this performance counter to compliment 
the information, but it should not be used as the only source of information. 





‘Number of Threads Alert 





<2xprocessor Acceptable value. The processors not causing an issue 

>=2 per processor Further Investigation required. Most likely the environment is suffering from 
a processor bottleneck during the analyzed period if system percentage of 
processor time is also high (caused by privileged or user time). 
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"The capture in Figure 2-7 shows 24 hours of performance logging and outlines 
how the processor queue length counter grows with the processor time when there isa 
processor bottleneck in the system. 


























Figure 2-7. Processor queue length analysis combined with processor time 


Memory 

The following issues usually cause a memory bottleneck: 
Insufficient RAM. 
. Amemoryleak 
^o System settings (BCDEdit exe settings). 


‘+ BizTalk Server throttling unnecessary due to a memory condition. 
when the server has still enough available memory. This could be the 
case ofa 64-bit host where the memory threshold is set to the default 
25%, In this scenario, the host can enter into a memory condition 
even though there are still plenty of memory resources in the server. 
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Memory\% Committed Bytes in Use 


This measures the ratio of committed bytes to the commit limit. In other words, the 
amount of virtual memory in use. This indicates insufficient system committed memory 
Ifthe number is greater than 80%. The obvious solution for this is to add more m 
ut adding a page f easing the existing one could be also options. 

You can use Table 2-1 as a guide. 











Table 2-1. Memory Committed Bytes in Use Thresholds 





Percentage Alert 





EA Acceptable value. There is enough memory to handle the load. 

>=70%4 and <B0% Warning. You do not want your server to be here for an extended period 
because if a peak load comes, applications can raise out of memory 
conditions. 

>=80% The amount of virtual memory in use might have reached the maximum. 
Consider ading more memory to the server if adjusting the page file does 
not solve the problem. 

>90% Ifthe page file is managed by the system, then it will automatically attempt 
o increase the page file. Therefore, if this counter reaches 95%, then it 
means that the page file is not being automatically increased or the page. 
file has been locked to a maximum size. Adjust the page file size, add a new 
one, or increase server memory. 





Memory\Available MBytes 


This measures the amount of physical memory, in megabytes, available for running 
processes. If this value is less than 10% of the total physical RAM, that means there is 
an insufficient amount of memor 





and that can increase paging activity. To esol 
this problem, you should first investigate the root cause and later decide If adding more 
memory will fx the issue, 

You can use Table 2-2 as a guide, 
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Table 2-2, Memory Available Mbytes Thresholds 





Percentage Alert 





E ‘Acceptable value. There is enough memory to handle operating system load. 


5% and <10% Warning. You do not want your server to be here for a long period because it 
a peak lad comes, applications can raise out of memory conditions. 





<10% ‘Critical. Peaks under high load might be acceptable during smal periods 
of time. Its situation happens frequently and in increasing rends, you 
should consider adding more memory to the server. 





Disks 
BizTalk not only stores data in a set of databases, but also elements such as configuration 
‘les, Windows Registry, and temporary folders. For obvious reasons, a bottleneck 
aliccting disk performance can have a significant impact on the servers. 

Ifthe system is running with insufficient physical memory, the operating sys 
make excessive use ofthe page fle, and that could eventually affect disk performance (if 
this condition lass for an extended period). 





to check whether the disks are 





You should use the following performance countes 
performing under the expected thresholds. 


^o Logical Disk (")\ % Free Space 
+ Logical Disk (*)V dle Time 

+ Logical Disk (*\ 6 Mg. Disk Sec/Write 

^o Logical Disk (*) 6 Avg, Disk See/Read 

+ Logical Disk (*) %. Disk Write Queue Length 
+ Logical Disk (*) t Disk Read Queue Length 


What is the difference between physical and logical disks counters? A physical disk 
is a LUN that is presented to the operating system, while a logical disk is a private unit 
within a specifie physical disk. Logical disks are presented as a volume with a drive letter, 
You can see this relationship using the disk management tool, as shown in Figure 2-8. 
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Figure 2-8. Difference between physical and logical disk counters 


Because the underlying hardware configuration for the disks ca 





be very 
‘complicated (virtual disks, new SAN hardware configurations, and more), if you do not 
know the physical architecture, it's better to start analyzing the LogicalDisk counter 
sets. This is because it is the closest measurement to what the processes will use. 


LogicalDisk\% Free Space 


"This counter measures the percentage offre sp 
disk. See Table 2-3. 





se ofa logical drive within the physical 





Table 2-3. Logical Disk Free Space Thresholds 








Percentage ‘Alert 
E ‘Acceptable value. There is enough disk space to handle the curent. 
capacity 





Warning. You do not want your server to be here for a long period because if 
a peak load comes, the server might run ut of space on that drive. 

<15% Critica if performance degradation is observed. In the context af a single 
physical drive, the performance can be impacted when low of free space. 
However, servers often use a SAN, which divides the data among multiple 
drives and may not sutter any performance degradation. 
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Keep in mind that this threshold might change for drives holding the BizTalk 
databases (especially under high load) because these databases are very transactional 
and the LDF files can grow extremely fast. 





Disk Idle Time Percentage 


"This counter measures the percentage of time the disk had no outstanding IO requests 
during the capture. Itis more important than it se 
indicator whether the disks are busy. See Table 2-4 








s because you can use it asan 


Table 2-4. Disk Idle Time Percentage Thresholds 





Percentage Alert 





An acceptable value. 
Warning. N disks are idle between 20% and 10% for a long period, it 
might indicate that the disks are getting very busy. Although performance 
degradation is not perceived yet, you should pay attention. 

<10% Critical. f the disk is queuing transfers and latency is alsa high, most lily 
the disk is overused. 








As you can see, if the percentage of disk idle time is less than 10% over long periods, 





you might need to check the disk queue length and read and writes latency to ensure 
performance is not affected. 
In Figure 2-9, we can see that when the BizTalk platform is processing more 





messages the idle time for the disk that holds the BizTalk Message Box database is 
trending to zero. That means that the disk is being used extensively. 
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Figure2-9, Disk Idle time and the Spool table 





Avg. Disk Sec/Read and Avg. Disk Sec/Write 


"These counters measure the average access time to perform the requested operation 
(read or write). 

The majority ofthe storage solutions of today implement very complex scenarios and 
use a wide range of configurations. Unless you are part ofthe storage team, keeping in 
mind elements such as LUN allocations, drive settings, and cabin distribution becomes a 
very complicated task. On top of that, modern virtualization technologies, and especially 
‘loud environmes 





s, relay entirely in virtualized storage solutions where the relationship. 
‘between physical and logical drives is almost impossible to disguise. For those reasons, 


these counters, also known as the disk response times, become an essential indicator of 





the disk performance, as they measure latency as accurate at the disk driver level. 
Asa general rule, you can follow the thresholds listed in Table 2-5. 
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Table 2-5. Normal Load: Avg Disc Sec Read and Writes Thresholds 





Response Time. Aet 





Acceptable value. Latency under optimum values. 





Warning This situation, along with increasing trends over the Spool 
able, could indicate that the sytem is getting very ose to experiencing 
a performance issue due to poor disk performance. 


>25ms (Critical. Disk response times are very likely to be affected. 





The previous recommendation works well for most of the non-aggressive BizTalk 
loads. However if your itis extremely low latency or very high throughput, 
especially for the disks holding the BizTalk database 
thresholds, as outlined in Table 26, 


environ 








jou might need to reduce these 


Table 2-6. High Load: Avg Dise Sec Read and Writes Thresholds 





Response Time Aet 





Acceptable value. Latency under optimum values 
Warning. This situation, along with increasing trends aver the Spool 
table, could indicate that he system is getting very close to experiment. 
bottleneck 





>15ms Critical. Business latency can be affected 





Keep in mind that when the disks hosting the BizTalk Server databases perform with 

reduced response times, BizTalk Server latency might also be affected, and because of 

that, all BizTalk latency counters might show dilated values as wel. In this situation, if 

your business is being affected, then disks might be causing a bottleneck in your system. 
igure 2-10, 





"The previous scenario is outlined in. 
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Figure 2-10. Average disk reads per second and BizTalk request response 


‘You can see that high read disk response is affecting BizTalk request-response latency. 


Avg. Disk Queue Length 


Do not confuse this counter with Current Disk Queue Length as they are calculated in 
‘completely diferent ways. Current Disk Queue length shows the number of requests 
that are outstanding on the disk, whereas the Avg. Queue Length shows an estimated 
calculation of (Disk Transfers/sec) (Disk sec/Transfer). 

Reads and writes are populated using the following two counters: 


+ Avg Disk Read Queue Length 
+ Avg, Disk Write Queue Length 


When no virtualization technology is used this counter is very reliable, and high 
values on this counter might indicate a disk bottleneck. However, with modern cabin. 
iving. In this scenario, 
LUNS can share hardware queues, and the cache has a significant impact on the values 


and disk virtualization, the output of this counter can be very dec 





E 
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ofthis counter. My recommendation is that you look for increasing trends over a period 
‘where disk response times are also high, See Figure 2-11. 























Figure 2-11. Average disk queue length and write operations 





In Figure 2-11, we can see that the Avg Queue Length shows increasing trends 
in traces A, B, and Calong with high writing latency (more than 15 ms and peaks of 
around 50). However, Inok at Figure 2-12. 
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Figure 2-12. Average disk queue length and write operations without issues 


In this case, you cannot identify a clearly increasing trend, and disk writing latency 
is under normal values. Most likely there is no writing latency issue here because writes 
are serviced on time. What you can do though is adjust the disk cache to improve the 
throughput, 


Network 


When a network bottleneck occurs, BizTalk will not be able to enlist new DTC. 
transactions against the destination SQL Server and that ar the end, will prevent 
processing messages. This situation can arise because of a malfunction of the 





networking components, network outage, or because the network is saturated due to 
outbound port exhaustion. You can analyze the following counters. 


n 
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Network Interface(*)\Bytes Total/Sec 


‘This counter shows the amount of data in bytes per second that go through the adapter. 
H computes receiving and sending bytes. As a general rule, more than 60% of the 
allocated bandwidth for that card should raise an alarm assuming that the network 
adapter is set for full-duplex mode. Since the counter is showing the information in 
megabytes, you can use the data in Table 2-7 as thresholds for different network card 


bandwidth. 





Table 2-7. Network Bytes Total Per Second Thresholds 








Bandwidth 60% Threshold 
100 megabits/s 7.5 megabytes/second 
1000 megabits/s 75 megabytes/second 
10000 megabits/s 750 megabytes/second 





the network is using 60% of the available bandwidth or more during a long period, 
then you should investigate what is happening with that specifie network card. 


Network Interface\Output Queue Length 


"This counter measures the length of the output packet queue, in packets. In an ideal 





situation, this value should not be greater than zero, You can use Table 2-8 as a guideline 
when analyzing networking bottlenecks. 


Table 2-8, Network Output Queue Length Threshold 








Number of Packets Effect 

a The network car is performing wel 

E Warning: Delays can appear, and al BizTalk latency counters start to be 
affected 

E Critical: Most likely BizTalk is being affected by high latency on ali the 
latency counters. Message publication and the dequeue process might 
also be affected, 





GHAPTEN2 PERFORMANCE ANALYSIS 


The example in Figure 2-13 outlines the effect af poor network performance over. 
the BizTalk latency counters. As packets are getting queued, BizTalk latency increases 
exponentially over time. 























Figure 2-13. Network output queue length and Biz Talk output latency 


Latency 


Measuring latency is probably one of the most important concepts when you want to 
analyze the performance of a BizTalk solution. Latency counters measure the time taken 
byall the BizTalk events since the adapter receives the message until the message is sent 
‘out. All latency counters are at host level. That means that Ifyou want to measure the 
latency ofa specific BizTalk element using these counters, you must isolate that element 
Into a specifically dedicated host. 

As seen in Chapter 1, BizTalk comes with settings orientated for high throughput 
scenarios. The following crucial concepts can change the engine behavior at message 
publication, dequeue, and orchestration processing: 
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+ Polling intervals (default of 500 ms)--Decreasing this setting reduces 
latency as BizTalk Server engine will consider this setting when there 
are new messages or orchestrations to process for a particular host. 





+ Receiving batch setting of the adapter Most of the BizTalk adapters 
receive information in batches. Although all adapters behave 
differently, this concept is very similar for all of them. The adapter 
retrieves messages from receive locations until the batch size is 
surpassed, or there are no more available messages atthe source. At 
this point, all messages of that batch are published to the Message 
Box database. Increasing the receive batch setting will also Increase 
the latency of individual messages as the engine publishes messages 
ina batch. On the other hand, setting the batch to 1 will decrease 
latency, as messages are published individually, and that process will 
be shorter than processing a larger batch. While there is not a specific 
recommendation for this setting, you can follow the guidelines in. 
Table 2-9 to tune it. 








Table 2-9. Receiving Batch Adapter Setting Recommendation 











Scenario Value Indication 

Low latency ‘Throughput can be affected 

Mixed Mone. This is the ideal value far the majority at 
scenarios. 

High throughput <max* Latency can be affected 





‘The max and min values are obtained during the performance test phases of the 
project when either the Message Box database or BizTalk Servers become a bottleneck 
due to the lack of hardware resources (or contention) and throttling based on database 
size is not occurring unnecessary (or disabled), 


+ ange message threshold setting—As seen in Chapter 1 this setting 
controls the number of fragments that are inserted in the Message 
Box database. Setting a proper value reduces or increases the latency 
of message publication, 
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+ Host separation policy—tsolating, receiving, processing, and. 
sending functionalities have a very positive effect on performance. 
dation isto isolate hosts by 





Even though the general recom 
functionality, you should only do it when latency is affecting your 
performance target level agreements. In such a case, if your hosts are 
running more than one BizTalk functionality and you detect that the 
latency performance counters are showing delays, you can follow the 
xt isolation advice, See Table 2-10. 








Note Chapter 4 digs into host separation policy recommendations based on 
several factors that are not related to latency itself, such as business requirements, 
type of application, and load, You can combine both techniques. 





Table 2-10. Basic Host Separation Policy Recommendation 








Counter. Recommendation 

High inbound latency You might consider creating a new host to dedicate all receive 
locations toit, 

High outbound and You might consider adding a new host to dedicate al send ports to it. 

adapter latency 

High outbound but not there is no adapter latency, then the problem might be at the dequeue 

adapter latency stage. Decreasing the pooling interval for that host might reduce latency 


while adding a new sending host might nat be useful in this scenario, 


High request-response You might consider creating a new host to dedicate all 
latency orchestrations to it 





Important: As you learned previously, all latency counters are at the host lev 
‘Therefore, you have to perform the changes only to the hosts that are showing the 
specific behavior 
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Note As every host instance consumes its own set of hardware resources, keep 
in mind that if you add more hosts to the BizTalk group, the server that runs that 
new host instance will consume more resources. As always, you have to test what 
is the maximum number of hosts for your hardware configuration. 





Latency Factors 


‘There are several factors that can affect the latency of your solution. The following 
le 





Load 


For obvious reasons, having a high load and peaks can increase the latency of your 
solution, especially f ardware resource consumption has reached a level where BizTalk 
start throttling conditions. 


Bad Disk Performance or Overused Disks 


‘The BizTalk engine uses storage in areas such as message publication, dequeue process, 
and configuration. Therefore, if disks are not performing efficiently, the BizTalk solution 
will struggle to process messages on time and latency will be directly affected. 


Throttling 


When throttling occurs, the engine applies a delay to the publishing and the dequeue 
processes. You will also see that latency counters show high: 
ra time will be reflected by the following 
throttling performance counters (BizTalk: Message Agent category): 





latency values because 





the engine is slowing down. The amount afe 


^ Message delivery delay (milliseconds)—Delay applied to the 
dequeue process. 

^ Message publishing delay (milliseconds}—Delay applied to the 
publishing process. 
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Complex Maps 


As maps can run at receive port, orchestration, and send port level, the time taken to 
execute has direct relationship to the latency of the solution. You will see how you can 
‘optimize map execution in Chapter 4. 


Complex Orchestrations and Custom Code. 


Orchestrations can also increase the latency of the solution as they can process code that 
itis out ofthe engine's control. You will see how to optimize orchestrations on Chapter 6, 


Size of the Message 


BizTalk uses messages in almost any components such as orchestrations, pipelines, 
maps, business rules, BAM, and the message engine. Therefore, the message size has 
substantial implications in all those areas. The BizTalk product does not have any 
limitation by design, but you should consider the message size in Initial stages of your 
solution development because, as message size growths, the messages per second 
rate decrease and general throughput declines (in some cases exponentially especially 
‘when executing maps). Keep in mind also that the larger the message size, the larger the 
latency you will observe across all the latency counters 

In Chapter 6, we will see how to optimize schemas for performance scenarios. 


Latency Performance Counters 


Lets review the latency performance counters: 
^^ Request-response 
+ Inbound 
* Outbound 


+ Outbound adapter 
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Request-Response Latency 


"This counter measures the latency in seconds since the message engine receives a 
message from the adapter until the response message is redirected back to the message 
‘engine, just before the message is sent to the original caller (this counter does not 
measure the ime of the final sending operation to the caller). See Figure 2-14. 








ck nthe eal — 








Figure 2-14. Understanding the request response latency 


"This counter generates the data per host level. you have more than one orchestration 
running on that host, the counter calculates average numbers fr all the orchestrations. 
‘Therefore, ifyou want to get data related toa specific orchestration, you have to create a 
‘new host and configure the orchestration to run under that dedicated host. 

Since orchestrations can take the subscriber and publisher roles, the engine can start 
a throtling condition that would affect both the message publication process and the 
message dequeue process, 
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Also ifthe orchestration dehydration process is affected by SQL Server performance 
‘or restriction settings latency will also be increased. Ifyou suspect that request-response 
latency is high because of issues with the dehydration process, you can check the 





Orchestratlons: Dehydrating Orchestrations counter. This counter shows the number of 
‘orchestrations that are currently in the process of dehydrating. Ifyou see an increasing, 
trend ov 





the period where request-response latency is detected, then you can 
Investigate if the bottleneck is on SQL Server side, as the database might not able to keep 
up with the dehydration load. 


Inbound Latency 


‘This counter measures the elapsed time since the engine receives a document from the 
adapter until the time itis published to the Message Box. As you know, orchestrations can 
also publish messages to the Message Box, but this counter does not reflect that time. 

the system initiates a throttling condition over the publishing stage, the Inbound 
engine is putting pressure on the 
publishing process. In this ease, the Message Publishing Delay counter will reflect the 
Increased time. 


Tatenc; 





counter will show higher latency values as th 





Outbound Latency 
‘This counter measures the elapsed time in seconds to complete the following stages (for 
an outbound operation} 

L. The message ls dequeued. 

2. Pipeline and maps are executed, 

3. Theadapter picks up the message and sendsitto the destination 

system, 
Mhe system initiates a throttling condition over the delivery stage, the outbound 

latency counter wil show higher latency values, as the engine is puting pressure on 


the dequeue process. In this case, the Message Delivery Delay counter will reflect the 


Increased time. 





ao 
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Outbound Adapter Latency 


"This counter measures the elapsed tin 





in seconds in an outbound operation since the 
adapter picks up the message and sends it ro the destination system. 

Note thatthe difference between the previous counter is that outbound adapter 
latency counter does not count the time spent during the Dequeue process while the 
outbound latency counter does, 


Latency Thresholds 


you take latency as an essential indicator when the system performance is degraded, 
‘what thresholds can you use for late 





y counters? That is nat an easy question to answe 
asit depends on lots of external factors such as business requirements, peak loads that 
‘can affect overall latency, and hardware configuration 

What is an optimal latency then? 








+The one that can be obtained when the business flow is not affected? 
All load within thar specific flow is going in and out under the agreed 
performance service level agreement (SLA). 

+ The one that can be obtained when there are no bottlenecks at 
BizTalk and SQL Server layer? 





In my experience, in most of the BizTalk environments where the personnel has 
aan awareness of the latency concept, they choose only the second option to define 
the latency thresholds, as in most of the cases the business team has not provided the 
performance requirements or in the best of the scenarios they are entirely obsolete. 
Anyway, in both cases, the value should be gathered/ verified from performance 
ing. That is the best approach, Ideally, y 
‘one of the following conditions are met first: 


tes 





ju will be stressing the testing platform until 


‘+The SQL Servers become the bottleneck 
+ BizTalk become the bottleneck 


+ Business Performance SLA is nat met for the Now 


ar 


CHAPTER2 PERFORMANCE ANALYSIS 


[the business team provides you the required latency, you are lucky! That isit, you 
got your latency thresholds, and you ean size you BizTalk. 
those values. But if not, you need to reach a point where you get a server bottl 
(BizTalk or SQL) gather the latency counter values, and fulfil Table 2-11. 





nvironment to perform within 
" 





Table 2-11. General Latency Thresholds 





Seconds Alert 





m Acceptable value. Latency under optimum values 
>= 140% and <¥-20% Warning. This situation, along with increasing trends over the 
Spool table, could indicate that the system is getting very close to 
experiment a bottleneck 
>= 120% and <x-10% Critical. Latency shows that the performance of the system is cose to 


being in a non-compliant scenario. Where x is the latency value when 
BizTalk or SOL Servers become the bottleneck. 





Let's illustrate this with an example: Imagine that you stress the testing server to 
a point where BizTalk processor time counter is close to 80% fora long period. That is 
indicating already a bottleneck at CPU level. You gather the request-response latency 
‘counter, and you get a value of 20 seconds. 

Now if you fulfill the latency table (x= 20), as shown in Table 2-12. 





Table 2-12. An Example Implementation of Latency Thresholds 





Seconds Aet 





<12 Acceptable value. Latency under optimum values. 

>=12and<16 Warming. This situation, along increasing trends over the Spool table, could 
indicate that the system is getting very close to experiment a bottleneck. 

>=t6and<18 Critical. Latency shows that the performance of the system is close to being 
in a non-compliant scenario 





m 
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You can use those latency thresholds to raise an alert when troubleshooting or 
monitoring. 





Important Remember that this procedure makes sense only when the business 
team has not provided the performance thresholds. On the contrary, you should 
use the provided ones and adjust the hardware resources to run under those. 
thresholds. 





Keep in mind that when there are increasing trends in all the latency 
related to outbound operations, you will also see that the Spool table is being affected 
by this fact. The Spool table holds references to messages that are alive in the system. 
As latency increases, those messages will remain in the Spool table for a longer time, 
and that can eventually rise a throttling condition based on database size or slow down 
the message engine as the Message Box database grows 











Throttling 


Without a doubt, the BizTalk throttling mechanism is one of the scariest toples for every 
BizTalk administrator. No one wants to see business users panicking, arriving at your 
desktop and saying in a solemn voice: "BizTalk is processing messages terribly slow! Can. 
you check what is happening?” 

At first glance, this mechanism brings with it a negative conceptions since when 
throttling enters into action, the system will process messages more slowly or will not 
process them at all. However, Microsoft designed this mechanism to alleviate pressure 
and prevent reaching a severe situation, 








Note Think of it this way: When this condition appears, something can stil be 
done to solve the problem. If the throttling mechanism were not implemented, the 
system would undoubtedly reach a state without a solution. 





Ifyou are a seasoned BizTalk expert, one of the first things you should probably 


check isthe status of throttling, because this often is the most likely cause ofthe 





slowness, 
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What Is Throttling? 


‘Throttling is the mechanism that implements the BizTalk messaging engine to reduce 
the rate at which messages are processed. That includes running any of the basie 
functionalities ofthe engine such as receiving, processing, and sending messages, 

Why does it need to slow down? When BizTalk is processing messages under high 
load, to prevent the system from running out of processor, memory and disk resources 
(on the database side), the message engine initiates a throttling condition. The engine 
increases latency in two areas: 








+ Message publication- Messages stored in the Message Box 





+ Dequeuing process—Messages retrieved from the Message Box 


‘This mechanism monitors the status of the following resources to verify if any of 
them exceed the established usability threshold: 


+ Consumed memory—The memory that both the system and the hast 
Instances are consuming, 


+ Number of in-process messages—Messages that are still not 
processed by any subscriber. This number is directly related to the 
current system load. The more subscribers with pending messages, 
the more messages are in the in-process queue. Messages that are in 
the internal memory queue are excluded from this calculation, 


+ Number of concurrent database connections established with the 
SQL Server instance that holds the BizTalk databases—This number 
is not accumulating over a host instance session. 





+ Number of threads—Processors assign tasks to threads. The throuling. 
algorithm ensures that the number of threads per processor does not 
exceed an absolute limit 


+ Rate of publishing delivery, or processing —When the relationship 
between the publishing and delivering rates is not balanced, BizTalk 
initiates throttling and applies delay to the function with a high 
rate, This one is the most common of throttling conditions, as 
systems usually receive and send messages in batches (or randomly) 
Rate in this context indicates messages per second. 
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+ Message count in Message Box database tables This number is 
computed by counting the number of rows in these tables within the 
Message Box database: 
+ Main queue tables for the host 
+ Spool 
+ Tracking tables 
any ofthese thresholds are reached, the throttling algorithm applies restraint to 
the publication of messages orto the Dequeuing process, depending on whether the 
message is recelved or sent, The severity varies depending on which condition has 
exceeded ls established threshold, as explained in Tabl 








Table 2-13, Severity of the Most Common Throttling Conditions 





Condition. Severity 





Memory threshold 
In-Process threshold 


Message count in database. 


5 
" 
Number of threads 3 
2 
Rest of Throttling conditions 1 





Arating of is the most aggressive severity and 1 is the least. The higher the severity, 
the longer the delay that is applied by the engine. 








Note This severity classification based on numbers is not used internally by the 
engine; it has been included for clarification reasons. 





At the configuration level, you can change the throttling severity using the BizTalk 
Administration Console, as shown in Figure 2-15. 
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Figure 2-15. Severity customization for the throttling mechanism 


As you can see in Figure 2-15, ifyou leave the default settings, the most aggressive 
throttling condition will be caused by memory situations. In this case, the engine applies 
a pressure 500% higher than a normal throttling condition. 

you have disk space limitations, it might be interesting to change the severity for 
the DB size 


s 


CHAPTER2 PERFORMANCE ANALYSIS 


Throttling Performance Counters 


The throtling mechanism could be overly complicated to analyze. Luckily, the 
engine uses a set of performance counters that can help you determine the following 
information: 





+ Gehe engine has started a throttling condition 
+ How long it’s been doing throttling 
+ How aggressive the condition is 


+ Throttling vital information 





When the engine throttles, it will populate several performance counters under the 
BizTalk Message Agent category: 
. Active instance count—Refers to th 
the in-memory queue. 


number of instances that are in 





+ Database session 





Number of opened connections to the Message 
Box database. 


+ Database size—Sums the number of messages in all host queues and. 
the Spool and tracking data tables within the Message Box (not the 
tracking database), 

+ High database session the engine enters into a throttling 
condition based on database size, this counter will show a value af 
‘one; otherwise, it shows zero. 


+ High in-process message count—Irthe engine enters into a throttling 


condition based on in-process message count, this counter will show 
a value of one, otherwise, it will show zero 





^o High message delivery rate—IIthe message delivery rate exceeds 
the message processing rate this counter will show a value of one, 
otherwise, it will show zero. 


^ High message publishing rate—If the message publishing rate 
exceeds the message delivery rate this counter will show a value of 
one, otherwise, it will show zero. 
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+ High process memary—One if process memory exceeds the process 
Virtual memory threshold, otherwise, i's zero. 


+ High system memory—One ifsystem memory exceeds the system. 
memory threshold, otherwise, it's zero, 


+ High thread count—Ifthe number ofused threads forthe host 
instances ofa specifie host in the server exceeds the number of 
threads per CPU setting, this counter will showa value of one, 
otherwise, it will show zero, 


+ In-process message count—Number of in-memory messages for this 
host 


+ Message delivery delay (ms)—If throttling condition is raised, this is 
the delay in milliseconds that the engine will apply to the dequeue 
process. 

+ Physical memory usage (MB)—Physical memory used in the server. 
“This counts even non- BizTalk Server processes 


+ Process memory usage (MB)- This is the maximum of the process 
working set size and the total space allocated for the page file for the 
process. 





As you learned previously, the engine can apply throttling to the publishing or 
dequeue processes. To reflect that situation, BizTalk populates the following state 
performance counters: 


+ Message delivery throttling state 
+ Message publishing throttling state 


Message Delivery Throttling State 


"This counter indicates whether the 
the following values: 





ngine is throuling the dequeue process. It can show 
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Value Description 





D The engine is not throtting 
1 Input rate exceeds the output rate 

3 High numberof messages in the in-process queue 

4 BizTalk host instance reaches the virtual memory threshold 

5 System memory reaches the system memory threshold 

s The host instance reaches the number f used threads threshold 


10 Throtting due to user override on delivery; occurs when you have changed the 
message delivery throttling settings for that host 





Input Rate Exceeds Output Rate (1) 


"This throttling condition can occur by high processing complexity, slow sending adapters, 
or poor system resources such as processor, memory or 1/0 (disk and network). 

"This is the situation where the message incoming rate exceeds the message outgoing 
rate using the following formula: Message publishing outgoing rateV* the specified Rate 
overdrive factor (percent) value. 

The rate overdrive factor (percent) parameter is configurable on the Message Publishing 
‘Throttling Settings dialog box and by default is set to 125%, as shown in Figure 2-16. 
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Figure 2-16. Publishing rate overdrive factor setting 
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For simplification, you can think that when the engine is publishing messages 125% 
faster than delivering, this throttling condition starts. 


1. The engine puts pressure on the publishing process 
2, High message publishing rate counter is set to L 


3, The message publishing throttling state duration counter is reset 
to reflect the new throttling condition. This counter is measured in 
seconds 


4. The message publishing delay counter ls updated with the delay 
time induced per message (or message batch), 


5. As the publishing process is delayed, you should see also that 
the inbound latency and request-response latency counters are 
affected 


High Number of Messages in the In-Process Queue (3) 


In Chapter 1, you learned how the dequeue process works. In-process messages are 
messages that have been retrieved from the in-memory queue and are not yet processed 
bby any subscriber (send ports or orchestrations) 

The in-process messages are configurable on the Resource-Rased throttling dialog 
box and by default are set to 1000. See Figure 2-17. 
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Figure 2-17. In-process message setting 
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1. “The engine puts pressure on the delivery process. 


‘You can query the in-process message count counter for that 
host to check the value and to look for increasing trends until the 
condition has risen. 

3. High in-process message count performance counter is set to 1. 

4. The message delivery throttling state duration counter is reset to 
reflect the new throttling condition. This counter is measured in. 
seconds 

5, The message delivery delay counter is updated with the delay time 
induced per message (or message batch). 

G. Asthe dequeue process is delayed, you should see also that all 
BizTalk latency counters are affected but the incoming latency 
that is related to publishing is not 

In my exper 
queue setting, Les see an example 

Imagine that you are receiving 5,000 messages simultaneously and that your 
business SLA is to process each request within one second, The first 1000 request will go. 
‘extremely fast but then BizTalk will start throttling as the in-process, and the in-memory 








a have to tune this setting in combination with the i 





memory 





queue counters values, will cause BizTalk to start throttling. From message 1,001 
latency is applied and it might happen that latency will decrease to a point where every 
individual message takes more than one second to process (remember that the throttling 
algorithm can apply up to 300 seconds to every message), 

In this scenario and optimum value could be 5,000 for both queues. Memory 





consumption will grow though so you need to increase the available memory of the 
server because ifnot, a throtling condition due to available memory will occur, and this. 
condition is the most aggressive one. 





Note Ifthe in-memory size ofthe incoming files is 500 KB, then 5,000 messages 
means around 2.5 GB of memory consumption for this process only 





‘You can have a similar situation when processing large batch files (more than 1000. 
messages per file). 
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BizTalk Host Instance Reaches the Virtual Memory Threshold (4) 


"This situation occurs when the BizTalk process reaches the threshold for virtual memory 
‘consumption, 

‘The process virtual memory setting is configurable on the Resource-Rased throttling 
dialog box and by default is set to 258; see Figure 2-18. 


Mary Usage 
Global phyical [] 





Figure 2-18. Process virtual setting 


the host instance enters in this state, the BizTalk engine will apply the highest 
severity condition, and the system will slow down perceptively. 

I the hosts are 64-bit, host instances can address up to 128 TB of virtual address 
‘space, In this case, tis recommended to increase this setting up to 100% which gives the 
process the potential to add the majority of system physical memory into its working set. 
Be aware though that in most scenarios, there will be more hosts and you do not want 
to enter into a situation where this host takes all the memory of the servers, blocking the 


wry resources for the rest of the hosts, 








‘System Memory Reaches the System Memory Threshold (5) 


"This situation occurs when the BizTalk process reaches the threshold for global physical 
memory consumption. The global physical memory setting is configurable on the 
Resource-Based throttling dialog box, as shown in Figure 2-19. 


Menon 








Figure 2-19. Global Physical memory setting. 
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By default, it's set to 0. This means that BizTalk will not evaluate the system memory 
‘condition, Ifyou change this setting and the host instance enters in this state, the 
BizTalk engine will apply the highest severity condition, and the system will slaw dawn, 
perceptively 

Do not change this setting unless all your host instances are 32-bit and you have 
an extremely limited amount of memory. IT your host instances ate 64-bit, the global 
physical and process virtual settings behave the same way, as 64-bit host instances can. 
potentially take up to 100% of the available memory of the server. 


‘The Host Instance Reaches the Number of Used Threads Threshold (9) 


Each process offers the resources needed to execute a program. A process has a virtual 
address space, running code, and a base thread priority. Each process is started with a 
single thread, but can create additional threads as is the case of BizTalk host instances. 

BizTalk host Instances load a minimum number of threads per CPU dedicated for 10 
and the rest far operations. These settings can be tuned using the BizTalk Administration 
Console, as shown in Figure 2-20. 


pd 





Figure 2-20. .NET CLR threads for host instance usage 
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Reducing the number of reads that the BizTalk Server engine use can improve. 
performance when processors are showing a high amount of context switching. 





Note The actual CLR is the one owning this thread's resources, but for the propose 
of this concept, these threads will be also counted by the throttling algorithm. 





You can think of these settings like a thread pool, When a host instance starts, it loads 
25 threads for /O (disk and network operations) and another 5 for the rest of the engine 
tasks. That means that, by default, each host instance wil reserve 30 threads “without 
doing anything" 

You can contol how BizTalk enters into a throttling condition by adjusting the 
setting Threads, using the BizTalk Administration Console, as shown in Figure 2-21. 
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Figure 2-21. Number of threads per CPU throttling threshold 


By default, this setting Is set to 0. That means that BizTalk will not enter into the 
throttling state based on threads consumption, Whenever you adjust this setting, keep in 
mind the host instances settings for the Net CLR, because if you adjust the thread setting 
forthe host toa lower value than the minimum values specified for host instance, all the 
‘host instances of that host will not process anything, and the engine will enter into the 
throttling state as soon as the host instance loads. 
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Throttling Due to User Override on Delivery (10) 





I you change any of the settings related to rate-based throttling BizTalk will enter Into 
this state. 
Message Publishing Throttling State 





"This counter indicates whether the engine is throttling the publishing process. It can 


show the following values: 








Value Description 





The engine is not trating 
Output rate exceeds input rate. 


D 
2 

4 BizTalk ost instance reaches the virtual memory threshold. 
5 System memory reaches the system memory threshold. 

6 


The number of messages in the Spool and tracking data tables exceeds the defined 
threshold 


9 — The host instance reaches the number of used threads threshold. 


11 Throtting due to user override on delivery This occurs when you changed the message 
delvery throtting settings fr that host, 





Output Rate Exceeds Input Rate (2) 


‘This throttling condition can occur by high processing complexity, slow sending 
adapters, or poor system resources such as processor, memory, or I/O (disk and 
network). This is the situation when the message output rate exceeds the message input 
rate using the following formula: Message delivering outgoing rate\* the specified Rate 
overdrive factor (percent) value. 

‘The Rate overdrive factor (percent) parameter is configurable on the Message 
Publishing Throttling Settings dialog box and by default is set to 125% (see Figure 2-22). 
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Figure 2-22. Delivery overdrive factor for the rate-based throttling condition 
For simplification, you can think that when the engine is delivering messages 125% 
{aster than publishing, this throttling condition starts, 
1. The engine puts pressure on the publishing process, 
2, High message delivering rate counter is set to 1. 


3, The message delivering throttling state duration counter is reset to 
reflect the new throuling condition, This counter is measured in 
seconds 
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‘The message delivering delay counter is updated with the delay 
ime induced per message (or message batch), 


As the delivering process is delayed, you should see also that the 
outbound latency and request-response latency counters are affected. 


BizTalk Host Instance Reaches the Virtual Memory Threshold (4) 


‘The same behavior as the delivery condition, but BizTalk will apply throttling to the 
publishing stage. 


‘System Memory Reaches the System Memory Threshold (5) 


‘The same behavior as the delivery condition, but BizTalk will apply throttling to the 
publishing stage. 


Throttling Due to Message Count in Databases (6) 


"The throtling mechanism is continuously monitoring the size of the following tables 


‘within the Message Box database: 





Host message queue size—It contains references to messages that are 
pending to deliver to a host and are not suspended, 

Host message state queue—The State Queue table saves the list 

of messages that have been processed by an instance but might 

be needed later This is used mainly in orchestrations when the 
developer creates a BizTalk message using a construction shape. 
Because the message might be used everywhere in the code, the 
message engine saves reference to it in the state queue table, 

Host message suspended queue—The suspended queue is where 
references to suspended messages ate stored. 

Spool table size—As you learned in Chapter 1, this table contains 
references forall messages in the BizTalk group. 

"Tracking tables—The TDDS service (subservice loaded into the host 
instance that has tracking enabled) moves tracking data and tacking 
events to the tracking data tables within the Message Box database. 
Later, the SQL Server job TrackedMessages_Copy 8izTalktsgBoxDb 
moves this data into the Tracking database. 
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Three settings regulate how the throttling based on database size works: 


Message count in DB— BizTalk will enter into the throttling state 
when any ofthe host message queues reaches this threshold. By 
default, W's set to 50,000, 


Spool multiplier If che number of messages in the Spool table 
hes the message count DB * the Spool multiplier setting, BizTalk 
will enter into the throttling state. 





Tracking multiplier—If the number of messages in the tracking data 
tables reaches the message count DB the tracking multiplier setting, 
BizTalk will enter into the throttling state. 





You can adjust these settings using the BizTalk Administration Console, as shown in 
Figure 223, 
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Figure 2-23. Message count in database settings 


"The most common situations when BizTalk enters into a throttling state due to 
database size are: 


+ Trackedessages Copy BizTalktisgBox0b is not running or is 
running slowly (the SQL Server Agent might be stopped), 


+ Subscribers are not processing messages from the in-memory queue 
due to he lack of resources. 


+ The number of suspended messages is high. 


+ Maximum sustainable load for the BizTalk group has been reached. 
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[the throttling condition has been raised due to Spool or tracking tables sizes, all the 
hosts across the BizTalk group will enter into the throttling state, even though the host 
responsible for throtling was just one. Why? 

Since the Spool table is shared across all the hosts in the group, the system throttling 
In this way could indicate that the maximum sustainable load for the system has been. 
reached, (Mixing data from all the host configured in the group; therefore, in this case, it 
does not matter what host is causing the throttling and the system will reflect this fact by 
entering throttling for all the hosts in the group.) The same concept applies to tracking 
tables (which are shared across all hosts in the group). 





‘The Host Instance Reaches the Number of Used Threads Threshold (9) 


The same behavior as the delivery condition, but BizTalk will apply throttling to the 
publishing stage. 


Throttling Due to User Override on Delivery (11) 


The same behavior as the delivery condition, but BizTalk will apply throttling to the 
publishing stage. 


Suspended Messages 


BizTalk stores messages in the Message Box database. Ifa failure occurs at the pipeline 
or orchestration level BizTalk suspends the instance. There are two types of suspended 
service instances: 


+ Suspended instances that you can resume. 
+ Suspended instances that you cannot resume. For example, ifan 
Instance is corrupt or there is a routing failure (RFR). 

BizTalk does not automatically remove suspended instances. Therefore, you have to 
resume or terminate them manually 

As reviewed in the “Message Engine" section, the engine creates a set of tables for 
‘each host within the Message Box database, One of the tables is called the suspended 
queue and is used to store all suspended messages. 
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Impact of Suspended Messages 


Each suspended message is stored in the suspended queue for that host. This action has 
several implications that can cause BizTalk to perform slowly: 


Spool table grows since it has references to those suspended 
messages. 

Latency and throughput. Each internal stored procedure that th 
BizTalk engine runs must “iter” and execute even more records 
than usual, and within in a 100,000 execution, it will have a negative 
impact, even more in a low latency scenario where pooling intervals 
settings are aggressively reduced. 








‘Throtling due to message count in database threshold is also affected 
since suspended messages are included in the message count in 
database calculation (because of the Spool and Tracking Spool table 
sizes). Throttling due to message publishing can occur even if BizTalk 
is experiencing low or no load, 


Monitoring Suspended Messages 


As suspended messages can impact performance negatively, implem 





ting a solution to 


monitor them becomes crucial. This can be done using the following methods: 


Performance counter data 
WMI classes 
BizTalk Health Monitor tool (BHM) 


External tools, such as system center operations manager and BizTalk 
360 (both tools will be covered in the monitoring section) 


‘The following methods are not detailed in the book, as Lassume you have mastered 


them: 


Reviewing the Windows EventLog 


BizTalk Administration Console 
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Performance Counters 
The following performance counters could be used to monitor suspended messages: 


‘+ Host Queue - Suspended Msgs Length and Documents suspended. 
Tracks the total number of suspended messages for a specific Host. 


+ Documents suspended/Sec. Tracks the current number of suspended 
messages per second for a specific host. 


Windows Management Instrumentation Classes 


The HSETS MessageTnstance WMI class ean be used to monitor and solve suspended 
messages instances. It loads message instances from the instances table within the 
Message Box database. Its most relevant properties are listed in Table 





Table 2-14. Properties of the MSBTS Messagelnstance WMI Class 








Property Description 
HostNane Name of he host hat published or consumed the message instance 
MessageType Contains the HessageType property 





Its most relevant methods are listed in Table 2-13. 


Table 2-18. Methods of the MSBTS. Messagelnstance WMI Class 





Property Description 





SaveToFile Saves the message context and message body part to a location 





The NSETS. ServiceInstance class loads all service instances from the instances 
table, The difference with the MSBTS_MessageTnstance class is that it shows th 
‘orchestration instances as well. Its most relevant properties are listed in Table 
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Table 2-16. Properties of the MSBTS Servicelnstance WMI Class 








Property Description 
Errorid The exception number that raised the error. 
Servicestatus Status of the service. It can be. 


16 (Completed with discarded messages)32 (Suspended not 
resumable}4 (Suspended resumable) 


SuspendedTime Time that the service was suspended, 








Hs methods are listed in Table 2 


Table 2-17. Methods of the MSBTS Servicelnstance WMI Class 








Method Description 
Resume Resumes the suspended instance 
Suspend Suspends an active instance 

Teminate Terminates a suspended instance 





BizTalk Health Monitor Tool 


Although the BHM tool will be covered in more detail in the monitoring chapter, 1 








include the suspended message s 
Once the BHM tool is configured and after you run an analysis, navigate to the 


ction here for your re 


Message Box Database section. If the platform has suspended messages, the tool shows 
analert like the one in Figure 2-24, 
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CRESS 





@ vastwoord 





Figure 2-24. Suspended messages alert in the BHM tool 


In this case, the alert is marked as a warning, but if the number of suspended 
messages reaches the 50,000 threshold, it would be marked as critical. You ean 
‘customize the default thresholds by changing the profile settings. 





Visual Basic Script to Deal with Suspended Messages 


"This scriptis intended to be asa tool to deal with BizTalk suspended messages, 
especially if the environment is suffering from a massive number of suspended 
messages. 

You can view a full description and how to use itin the TechNet Wiki article I wrote, 
called "Visual Basie Seript to Deal with BizTalk Suspended Messages" and found at 
http: //social. technet.nicrosoft. con/wiki/contents/articles/28157.BizTalk- 
script-to-deal-with-suspended-nessages-vbs aspx 

Or you can find tin my personal blog here: http: //blogs.nsdn.con/b/anantaras/ 
archive/2014/10/26/ inproved-visual-basic-script-to-deal-with-BizTalk- 
suspended-nessages aspx. 

"The code is published in MSDN Code Gallery, and you can download it from 
https://gallery.technet .microsoft.com/\Visual -Basic-script-to-8542997. 
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Tracking 


Tracking is the BizTalk feature that makes it possible to track everything that has 
happened in the platform during a specified period. The tracking information can be 
classified into two types: 


"The tracking feature is completely decoupled from 


Data BizTalk can track information related to the message 
content, promoted properties, partner data, schemas, and routing 
information (since tracking allows you to trace the path ofa message 
as it is routed through the BizTalk, enabling tracking can be useful for 
troubleshooting errors) The Group Hub can display error codes and 
routing states fora message so that you can troubleshoot errors in 
real time. 





Events This type of tracking is related to the time that the 
information is tracked than the information itself. BizTalk can track 
events such as 


+ Thestartor end ofa service 





+ When messages are sent ar received. 


+ Whena pipeline starts or ends 





+ When an orchestration stars or ends 


+ The execution of each shape in an orchestration 





ic publishing and subseribe 


mechanisms, so you can adjust it without affecting the business functionality. 





‘Warning! tis true that, from a functional viewpoint, this feature detaches 
from the messaging engine. However, you have to be careful when changing the 
tracking settings, because if you increase the amount of tracked information, the 
level of performance requirements will increase, affecting to the SQL Server that 
hosts the BizTalk databases. If the tracking database is not isolated in a separated 
SQL Server instance, it could also affect the Message Box database. 
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"The tracking configuration is adjusted using the BizTalk Administration Console. You 
‘can track the following types of data or events: 
‘+ Inbound and/or outbound event data—For example, message ID, 
and start and stop times for the artifact. 


‘+ Inbound and/or outbound message properties—For example, 
general and promoted properties for each message that the artifact 





processes 


‘+ Inbound and/or outbound message bodies and parts— For example, 
body and parts for each message that the artifact processes. 


^o Orchestrations- Execution data for orchestration shapes 


Tracking Performance Counters 


BizTalk exposes the following performance counters related to the Tracking feature: 


+ Tracking data size This counter reflects the size of the tracking data 
in the Message Box. As SQL Server's purging jobs move data from 
Message Box to tracking database, any Increasing trend over a period 
will indicate that there is probably a bottleneck in the tracking feature 
and the Message Box database will grow. This will eventually raise a 
throttling condition based on the database size and the system will 
process messages very slowly. 


‘+ Tracked Msgs Copy (Purge Jobs) and Tracking Spool Clean Up 
(Purge obs) These two jobs show the time that it took both jobs to 
complete the task during the last run. Increasing trends over a period 
will also indicate that the system has a tracking bottleneck. Ifthe 
tracking data size counter is also growing exponentially while these 
Jobs are taking longer to complete, you should suspect a performance 
issue on SQL Server (most likely due to IO contention and/or CPU 
processing). 
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Summary 


In this chapter, you learned several common performance analysis techniques that 
will help you identify problems related to performance. Using these techniques, in 
‘combination with all the knowledge detailed for the most common counters, should give 
youa great toolset to start analyzing performance issues, 

Several books about Windows performance can be written only on processor 
performance analysis, so keep in mind that this book provides basie guidelines that will, 
orientate you along the right track. 
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Performance Tools 


In this chapter, you learn how to use the most common performance tools that will help 
you find bottlenecks and resolve performance issues in your BizTalk Server environment. 
‘The tools that you are going to learn about are 
+ Performance Monitor 
^o Performance analysis of logs (PAL) 


+ LoadGen 


Performance Monitor 


The Performance Monitor isan essential oo! to analyze performance counters. 
Iris included in every Windows server installation and in Windows 10. Windows 
Performance Monitor uses performance counters, event trace data, and configuration. 
information combined into Data Collector Sets for viewing performance data in real 
time or offline. I will be your primary out-of-the-box tool to capture and analyze your. 
BizTalk Server platform, The Performance Monitor tool gathers the following data: 











+ Performance counters are measurements of system state or activity. 
‘They can be included in the operating system or can be part of 
individual applications. Windows Performance Monitor requests the 
current value of performance counters at specified time intervals 





* Eve 





ce data is collected from trace providers, which are. 
components of the operating sys 





or of individual applications 
that report actions or events. The output from multiple trace 
providers can be combined into a trace session. 





^^ Configuration information is collected from essential values in the 
Windows Registry. Windows Performance Monltor can record the data 
ofa registry key ata specified time or in intervals as part ofa log file. 
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Setting Up a Performance Counter Capture 


The best way to learn how to do this is practicing, In this section, you learn how to create. 
your performance capture using the Performance Monitor tool. 

"This exercise uses the BizTalk Server performance counters template that you ean. 
find in the companion fles for Chapter 2, located at C: MPRESSVChapter2 Verfean: 








^o Perfmon BizTalk Server tenplate vanl 
‘+ Perimon SQL Server template xml 


Follow the sa 





instructions for the SQL Server template 





Note The instructions provided in this section are meant to capture performance. 
data on an as-needed basis, in the case of a reactive scenario or to asses an 
environment in a specific situation such as a health check or during application 
performance testing. f you want to capture the performance data daily, you will 
need to set up the performance logging for circular capture. 








Prerequisites: Ifthe current user is a member of the Performance Log Users security. 
‘group, then you will have access to the New Data Collector Set Wizard, but will not be 
able to finish the wizard unless the user has administrator rights or ifyou specify an 
account that has administrator rights. 

Follow these steps: 


L OpenPerfuon.exe by typing perfnon in the Windows Server 
search box, as shown in Figure 3-1 
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Figure 3-1. Running the Performance Monitor tool 


2. By default, you can find the tool in the following folder: 
C:\ProgranData\Microsoft\Windows\Start Menu\Prograns\ 
Adninistrative Tools 
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3. Right-click User Defined Data Collector Sets and choose 
New > Data Collector Set, See Figure 3-2. 
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Figure 3-2. Creating a neu data collector set 


ue 


CHAPTERS PERFORMANCETOOLS 


4. TypeBizTalk performance as the name of the Collector Set. Then 
select the Create From a Template option and click Next, as shown 
in Figure 33. 
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How would you like to create this new data collector set? 
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Figure 3-3. Setting the data collector set name 
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E 


In the template selection window, click on the browse button and. 
locate the template files downloaded from the Apress site. This 

is the BizTal XL Tenplate. xn or SOLXMLTenplate. xml file. See 
Figure 3-4. Click Next. 
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Figure 3-4. Browsing for a template option 
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6. After you select the template files, the confirmation window in 
Figure 3-5 appears. Click Next. 





€ conte rent Cock. 


Which template would you ie to use? 
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Figure 3-5. Select the template confirmation 
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7. Specify the location of the output BLG file. By default, it is using 
XsystendrivettPerfLogs MdninVBizTalkPerformance. See 
Figure 3-6. Click Next. 
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Where would you like the data to be saved? 
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Figure 3-6. Store location for a data collector set 
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2. Click Finish. See Figure 3-7. 
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Figure 3-7. Finishing the wizard for a data collector set 
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9. Right-click the already created data collector set and select 
Properties as shown in Figure 3-8. 
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Figure 3-8. Accessing the properties of the data collector set 
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10. Inthe Properties window, go to the Schedule tab and click the Add 
‘button, as shown in Figure 3-9. 





Figure3-9, Adding a new schedule 
1L. In dhe Folder Action window, specify the following values 
(see Figure 3-10): 
à. Beginning date: You have to decide the best starting date. 
b. Expiration date: 2 Days after the beginning date. 


Start time: Make sure to set up a high load time frame. 
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Figure 3-10. Schedule time frame 


12. Click the OK button, 


13. Within the Properties window, go to Stop Condition tab and 
enable the Overall Duration checkbox. Specify one as the value 
and choose days as the unit. Click OK to Finish. See Figure 3-11. 
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Figure 3-11. Stop condition by overall duration 


14, Adjust the right sample interval This value controls how often the 
tool captures the information of the selected counters. You can 
establish the following intervals: 


a, Seconds 
b. Minutes 
co Hours 
d. Days 

e Weeks 


The interval setting is much more important than it seems since adjusting the 
appropriate value depends directly on the scenario you want to capture. 
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These are the factors that you should take into account: 


+ Reason for the capture. If you want to analyze a specific issue, you 
should specify seconds as you want to have the most data available 
during the period when the problem is happening. II you are going to 
measure the overall health of the platform, you should set this value 
to minutes and capture the information during 24 hours minimum. 
In this case, you can verify ifyour system is running under acceptable 
thresholds. 





+ BizTalk Messages processed per interval. If your platform is handling 
only one message per minute, you should setup 1 minute as the 
sample interval. On the other hand, ifthe platform process 20 
messages per second, you should choose seconds. 


+ Size of the output BLG fle. The final size of the BLG data is directly 
related to the system load, the interval value and the number of 
instances that the counter has. The more load the system has and the 
smaller the interval, the larger the resulting data. In my experie 
ifyou set the range to 2 minutes for a 24-hour capture, the output file 
will have a length between 40 and 80 MB. Therefore, by reducing the 
value toa few seconds, the output file will grow exponentially and the 
analysis will become more complicated, as you will have to analyze a 
larger volume of data. 








Follow these steps: 


L Right-click the Performance counter capture, as shown in. 
Figure 3-12. 


Rl Monitoring Tools 
E$ Data Collector Sets 
P BiTalkPerformance 
JR Sytem 


Figure 3-12. Access to performance counter properties 
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2. Inthe Properties window, set the Sample Interval as required. 
In this example, set it to capture performance da 
minutes. See Figure 3-13. 





every two 





Figure 3-13. Sample Interval adjustment. 


3. ClikOK. 


4, View as report. In some scenarios, itis beneficial to present 
performance information in report view. The first time you open. 
a BLG file, the Performance Monitor tool loads all the counters. 
See Figure 3-14. 
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Figure3-14. Dense Performance capture 


5, As you can imagine, the amount ofthe information displayed 
makes it almost impossible to diagnose a specific issue, You can 
switch to report view and then locate the counters that show 
abnormal values, as shown in Figure 3-15, 





Figure 3-18. Report view representation of the counters 
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6. In this case, this environment is queuing lots of threads an 
average. You can then select only this counter and analyze what 
happens during the capture, as shown in Figure 3-16. 

















Figure 3-16, Processor queue length graph detail 


"This server is queuing threads with very high peaks. You should investigate the issue 
by adding processor usage counters. 


"To view the information as a report, follow these steps: 
1. Select the report option view fram the toolbox bar, as shown in Figure 3-17. 
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Figure 3-17. Access to the report view 
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2. The tool switches to the report view. Look for abnormal values and 
filter the log as desire. 


The Performance Analysis of Logs Tool 


The performance analysis of log tool (PAL) helps you find performance issues in a 
performance logging capture. The tool analyzes the captured data against a set of pre- 
defined threshold fles for the majority of Mierosoft products. It hasa user-friendly 
interface that allows you not only to set up the analysis but also to edit or create your 
threshold fle with custom rules. The output of the analysis is an HTML report that 
contains a list of alerts that are generated when a performance counter value reaches the 
pre-defined thresholds. 

‘The tool was originally maintained in Codeplex, but it has been recently moved to 
GitHub. You can download the tool at https: //github. con/clinthuffnan/PAL. 

‘The installation of the tool is a very straightforward process, so we are not getting 
into the details. Download the tool and follow the Wizard. 











Creating a New Analysis Using PAL Tool 


In this section, you les 
‘captured previously using the perfmon tool. Although the instructions provided here 
will suit the majo 
discussed, 


n how to use the PAL tool to analyze a BLG fle that has been 








y ofthe scenarios, itis recommended that you explore all the options 
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Follaw these steps 


1. Once the PAL tool loads, click Next on the welcome screen, as 
shown in Figure 3-18. 
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Figure 3-18. The PAL tool welcome screen 
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2, Inthe Counter Log section, browse for the location ofthe BLG 
file that you have captured using the perfmon tool and click Next 
(see Figure 3-19). 
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Figure 3-19. Counter log path 
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Notice that you have a Date and Time Selector. Sometimes the BLG files are 
‘enormous, or you want to analyze what happened ata specific time range. You can use 


this option to filter the captured data by a beginning time and an ending time. 
E 


In the Threshold fle section, select the Microsoft BizTalk Server 


2006/2009/2010 option from the drop-down box (see Figure 3-20) 
and click Next, 


[rom 





[rm 


tte 











Figure 3-20. Selecting threshold file 
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Do not worry if BizTalk Server 2016 is not listed. Most of the BizTalk performance 
counters remain unchanged throughout all the produet versions. 

4. In the Questions section, navigate through all the available 


See Figure 3-21. 


questions and answer them according to the analyzed server 
(not tothe server where you are running the PAL tool), Click Next. 
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Figure 3-21. Answering the platform questions 
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5. Inthe Output Options section, select the desired Analysis Interval. 
AUTO is the default value, and it will create 30-time samples. I 
‘you wanta higher level of detail, you can increase this interval by 
selecting a time using the drop-down box. Increasing this value 
will cause the analysis to take longer to complete. See Figure 2-22. 
Click Next. 
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Figure 3-22. Selecting an analysis interval 


6. Also, notice the Process All of the Counters in the Counter Log(s) 
checkbox. This option is attractive if your BLG files contain. 
counters that do not have a threshold defined in the threshold 
file. It could be especially useful if your BizTalk applications are 
creating custom performance counters, and you want to inelude 
them in the final report. 
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7, In the File Output section, locate the folder where you want the 
tool to create the output HTML fle, See Figure 3-23. Click Newt 
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Figure 3-23. Setting the output options 


A. Notice that there is the option to generate an additional XML file. 


"This is particularly useful if you have a custom tool that reads 
those XML files. 
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3. Click Nest in the Queue section. This section will show you the 
analysis that the tool will run when you complete the wizard. As this 
is the first analysis, you will get something similar to Figure 3-24. 
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Figure 3-24. Viewing the queue 


10. In the Execute section, you have three options: 
Execute: It executes the current analysis and all of the queued 
analysis now. 


‘Add to Queue: It adds the current analysis to the queue and the 
wizard will start again from the Counter Log section to add a new 
analysis 

Execute and Restart: It executes the current analysis and will 
restart the wizard again so you can run another analysis in 
parallel This option is the preferred one when you havea 
compelling workstation, as PAL will run all the analysis in parallel. 
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1L. Select the desired option and click Finish, as shown in Figure 3-25. 
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Figure 3-25. Setting the execution options. 
12. Notethe threading option. By default, the tool detects the number. 


of cores in your computer and runs using the same number of 
threads, 


a 
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Now the tool creates an inlin 





PowerShell script that does all the 
hell window shown in Figure 3-26 appears. 





magle. The Power 








Figure 3-26. Status of the analysis 


n. 


Here you can see the status ofthe analysis. When the toal is about 
to complete the analysis, you wil see the text "Generating HTML 
Report... Depending on the size ofthe BLG, this operation can 
take from a few seconds to several minutes, Once the report 





is generated, the tool loads it automatically using your default 


Internet browser. 
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Understanding the PAL Output 


The first area of the 





eport contains an Alerts section. Here you can see all of the issues 
found during th 


Analysis of "DataCollectorO1.blg" 
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Figure 3-27. Alerts by chronological order section 
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‘You can click in the links to go toa specific time frame. Il you scroll down through 
the report, you can see see all of the detailed analysis for BizTalk Server and the rest of 
performance counters, as shown in Figure 3- 








Figure 3-28. ‘The rest of the analysis 
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Any issues are highlighted, and you can click on them to see the details. For instance, 
Ifyou click on the spool table analysis, you will get the spool table analysis graph, as 
shown in Figure 3-29, 
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Figure 3-29. Spool table analysis graph 
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The associated alerts are shown in Figure 3-20. 
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Figure 3-30. Spool table analysis alerts detail 





Using LoadGen to Test the Environment 


In this section, you learn how to run the LoadGen tool in your environment. While in 
Chapter 4, you will learn how to test your environment using Visual Studio this tool 
‘comes in handy when you are a BizTalk Server administrator, and you are not familiar 
with Visual Studio testing tools. 

You can download the toolat https: / /moi.nicrosoft. con/download/details. 
aspx?ide14925. 

Installation ofthe tool is very straightforward, and it does not have diferent 
dependencies than the BizTalk Server instalation. Install the tool following the wizard. 
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The LoadGen Configuration File 


LoadGen uses an XML configuration file to simulate loading y 





(our environm 





<LoadGenFranenork> 


‘<ConmonSection> 
<LoadGenVersion»2</LoadGenVersion> 
<OptinizeL initFileSize>20480000</OptimizeLinitFilesize> 
<NunThreadsPerSection»5</NumThreadsPersection> 
<SleepInterval>200¢/SleepInterval> 
<LotSizePerinterval>25</LotSizePerinterval> 
<RetryInterval>10000¢/RetryInterval> 


<Stoplode Mode="Files"> 
‘<NunFiles>2000¢/NunFiles> 
Total Time>36000</TotalTine> 
</Stoptode> 


Transport Nane="FILE"> 
<Assenbly»FileTranspart..dl1/FileTransport .FileTransport</Assenbly> 
Transport» 


</Comonsection> 





Section Nane="FileSection 
SrcFilePathoC: \Users\Adninistrator\Docunents\800K\BooksSolution\ 
Files\FFBooksOrder. txt</SrcFilePath> 
<OstLocation> 
«Parameters» 
<DstFilePath>C:\Users\Administrator\Docunents\BOOK\BooksSolution\ 
Ports\IN\</DstFilePath> 
</Paraneters> 
</DstLocation> 
</Section> 


</LoadGenFranework> 
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‘Table 3-1 shows a list ofall the most common configuration settings for a 


performance load. 

















Table 3-1. LoadGen Configuration Settings 
Setting Description Value 
Optimizelimitkilesize ifthe file is larger than the value here, LoadGen ¶ 20480000 

wil not use that fle. This is to avoid ou 

memory conditions. 
WumthreadsPerSection The number of threads that the tool willuse.The — S 

best value here is to set it to the number of cores 

in the running server. 
SleepInterval Time in miliseconds between test cytes. 200 
LotSizePerinterval Number of fles copied per test 25 
RetryInterval ithe test could not be done, LoadGen willrety — 10000 

after 10.000 miliseconds. 
Stoptode ‘You can choose the stoping condition of the test — Fles 

‘cycle by assigning this setting. Supported stopping 

modes are: 

A) Files. LoadGen stops when the number fies is 

reached, 

B) TotalTime. LoadGen stops when the test duration 

reaches the total time value. 

You can use bath and LoadGen will sop where any 

f those conditions are met. 
StopModeNunFiles Number of les to stop the test oye 200 
StoptodeTotalTine Tota duration to stop the test cycle. 36000 

(continued) 
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Table 3-1. (continued) 





Setting Description Value 





Transporthane Transport used to test the scenario. Curently File 
LoadGen supports the following out-of-the-box 
transports: 


A) Fie 

BTTP 
C) MOSeries 
ys 
Bso 
AWE 
Gcr 

FileTransportDLL Location of the fle transport DLL. LoadGen wil 
use this assembly to emulate the adapter protocol 
Al the assemblies are located at C:\Program 
Files (x86)\LoadGen\Bins. 


ScrFilePath Instance that will be used t test Test instance 
DstFilePath Here is where the tool wil drop the message Receive 
instances. location path 





Testing a Solution 


In this scenario, you are going to test the book orders solution provided in the Chapter 2 
folder. 
Follow these steps: 
1. Install the application using the MSI file located at C: MPRESSV 
Chapter2\BooksSolution\BooksOrders\nsi. 


2, Start the application. 


3. Test that the application works. Using the Windows Explorer, 
navigate to the C: \APRESS\Chapter2\BooksSolution\Test folder 
and run the Send1BookOrder bat file. 
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the application is installed successfully, you should have an 
‘output file located at C: \APRESS\Chapter2\BooksSolution\ 
PortsVOUT. 





5. Open a command prompt and navigate to the BINS folder where 
you have installed the LoadGen tool, Ifyou choose the default 
location, itwillbeat C:\Program Files (x86)\LoadGen\Bins. 


"The bookOrders solutions comes already with a LoadGen pre-configured file, and it 
is located at C: \APRESS\Chapter2\loadgen\Send2000F:iles.xn1. 
‘Type the following instruction at the command prompt: 





Program Files (x86)\LoadGen\Bins>LoadGenConsole.exe "C:\APRESS\Chapter2\ 
JoadgenVSend2000Fi les xnl " 


‘The process will start and the tool will show the screen in Figure 3-31. 














Figure 3-31. LoadGen output. 
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6. Wait until the process finishes and review the output folder and 
the BizTalk Administration console to check that everything runs 
smoothly. 





Now that you know how to use LoadGen tool, you can use Performance Monitor to 


‘capture all the BizTalk Server relevant performance data. Then later, you can use the PAL 





tool to analyze what happened during the test. 


Summary 


In this chapter, you have learned about the most common tools you will need to 
you should capture the performance 
counter information using the Perfmon tool and then you can pass through the output 


‘asses your BizTalk Server environment. First 





BLG file to the PAL tool to accelerate the discovery of performance issues. PAL will 

narrow the whole capture to the points of interests, If the PAL tool does not provide an 

overwhelming conclusion, you can use perfmon again and open those specific moments 

ofthe BLG fle to drill down on the issue and apply the techniques gained in Chapter 2. 
Congratulations, you are ready to jump into the BizTalk Server optimization. 

chapter, 

performance and to adjust configuration settings for very specific situations. 





here you will earn how to optimize the BizTalk Server platform to boost its 
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CHAPTER 4 


Optimizing the BizTalk 
Platform 


Probably one of the most missed tasks when designing BizTalk Server solutions ls the 
specification phase. Most of he integration development focuses on providing the 
business functionality without taking into consideration the impact on performance 
‘once the application goes live 

"This is especially rue for BizTalk Server, as per design, the platform operates ruled by 
the publishing and subscribe architecture thus, all BizTalk Server messages flow through 
BizTalk Server message box database. Additionally if you enable tracking and business 
activity monitoring, performance factors such as hardware sizing, message size, and the 
number of transactions (among others) becomes essential, Therefore, there is something 
you must keep in mind before starting a BizTalk Server project: The solution you are 
coding might not be the only one running in your production group. Applications 
compete with available hardware resources. Consequently, this should be your mantra 














when approaching a new project: 


Performance! Performance! Performance! 
Testing! Testing! Testing! 


During the initial phases of the project proper design and formal testing procedures 
willgh 
designing BizTalk Serv 
hardware estimation. Unfortunately, no systematic process precisely estimates the 





you an inital idea of the hardware sizing Ifyou have vast experience in 





architectures, intution could also guide you to an initial 


platform size for any BizTalk Server application. BizTalk Server is usually the core 
integration system within the organization, and it can potentially integrate a large 
variety of business implementations that run with their performance specifies, 


A Mitaras, Ta Server 206, tp doom 10107 A711 M2 a9 6 
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So, while an estimate based on existing experience provides a good starting point for 

planning purposes, the final size ofthe system will most certainly need to be adjusted 

along the way through a very structured testing procedure and good design principles. 
Tn this chapter, you learn how to: 


+ Categorize your applications based on a concept that is called 
application priority levels, 


+ Optimize BizTalk and SQL Servers based on those levels. 
. Use WMI and PowerShell to automate performance settings. 


+ Document application performance requiren 





+ Use that documentation to size the Message Box and Tracking 
databases. 


Assigning Application Priority Levels (APL) 


Iis good practice to categorize applications based on priority levels from a business 
point of view and the rest of the BizTalk Server factors that are relevant to solution 
architecture. By doing so, you will be taking crucial decisions during all phases of the 
project. The values used in this book are based on personal experience, but you should 
overwrite them to fit the project needs. 

APL is calculated by combining five essential elements that are directly related to 
SLA definitions 


+ Release stage level (RSL) 
+ Business priority level (BPL) 
+ Highavailabili 





y level (HAL) 
‘+ Number of transactions level (NTL) 


‘+ Performance behavior level (PBL) 


me 


CHAPTERA OPTIMIZING THE BIZTALK PLATFORM 


You can use the Description field in the application properties section to document 
the application priority level and any other relevant information, as shown in Figure 4-1. 
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Figure 4-1. Documenting the application priority level using the BizTalk 
Administration Console 
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Release Stage Level (RSL) 


This level represents how mature the solution is, or in oth 
See Table 4-1 





words, how stable itis. 


Table 4-1. Common Release Stage Levels 





RSL Description 





9  Fistrelase The application is in its first release version. New bugs can arise, and a 
high number f suspended messages can occur 

8 Unstable release Applications are now performing for a while, but there are stil some bugs 
that are not fied yet. Suspended messages can occur frequent. 

7 Unstable release Al known bugs are fed. Tracking is enabled at the orchestration level to 

(performance) find performance bottlenecks. No message body tracking is enabled at 

this stage, 

6 Stable release The application is now stable and running without known issues for a 
while now. Tracking is disabled, and normal hast settings are applied. 





Business Priority Level (BPL) 


This level should be agreeable to the business decision makers, as they are the ones 
‘who know the business. It is imperative that both parts understand the implications of 
this level because it will have a significant impact on early stages of the project and can 
Increase the cost of the platform, 

In BizTalk Server, we 


where any ofits components (equipment, personnel, process, procedure, software, and 





can define a mission-critical solution as an integrated system 





so forth) are essential to the business operation. Failure ofany of these elements usually 
results in a severe impact on business operations that can even influence the image of 
the organization. 

Examples of mission-critical systems are 


+ Anonline banking system. 
+ Aircraft operating 
+ Blectrie power systems 


+ Electronic bill applications 
um 
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^o Health care systems. 
+ ATMsystems 


"The majority of mission-critical solutions must run 24x7 without downtime. 
Performance is especially vital, as these implementations operate under very restrictive 
performance requirements for individual transactions. 

Additionally, these systems should remain very secure to prevent malicious attacks 
‘and to protect the data that goes through the BizTalk Server databases. 

‘As BizTalk Server is usually deployed to host mission-critical solutions, you will need 
a solid foundation on the performance factors that make a difference. 

"The definitions on this section are based on common customer deployments all 
around th 
Table 4-2, 


e world. Use them as an inspiration when categorizing your own solutions. See 





Table 4-2. Common Business Priority Levels 





BPL Example Definition 





9 — Mission-critical solution level 1. Organization operations are affected at a global scale, 
Failure of this application causes significant failures to other applications. The application 
has particular performance requirements. Zero downtime. Typically involves long running 
processes and very low latency scenarios. Performance degradation is not accepted 
(especialy under high load Usually, these applications are isolated in a dedicated BizTalk 
Server group, since even the administration team is frequently dedicated to operate these 
platforms 

8 Mission-critical solution level 2. The application is significant from the business point of 
view, but failures do not affect other applications, The application has particular performance 
requirements. Zero downtime. Performance degradation is not accepted (especially under 
high load 

7 Important application level 1. Major failure itis an issue. Performance is stil a fundamental 
requirement; an application can have specific performance requirements. Zera downtime 

6 Important application level 2 Functionality should be granted at any time but it the 
application run below the performance SLA during certain times is not an issue. The importance 
here is that al transactions are completed successully The business accepts downtime. 

(continued) 
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Table 4-2. (continued) 








Example Definition. 





5 Medium priority level. Intermittent failures are accepted as the consumer can resubmit 
the message. The Destination system or the integration layer can handle dupicated/missed 
transactions accordingly. The business accepts downtime. Performance degradation for a 
standard period is accepted as usual (even under high load 

4 — Low priority level. t the application fails, it wil not impact the business operations. This 
application does not have any specific performance requirement. The application can be 
manually stoppediresumed without any consequences at any ime. Performance degradation 
is accepted as normal 





High Availability Level (HAL) 


‘This level represents the high availability requirements for the solution. See Table 4-3. 


Table4-3. Common High Availability Levels 





HAL Description 





9 High availablity is required. Zero downtime. 

8 — High availabilty is required. Downtime is accepted to fix issues (debugging, stopping 
instances). 

7 — High Availability is required. Downtime is also accepted for deployment. 

6 Even though high availability is required, it cannot be applied due to platform restriction ar 
adapter behavior. Downtime is accepted. 

5 High Availablity is not required. 





Transaction Levels (TL) 


"This definition is directly related to the business performance requirement. It is not 
calculated based on specific hardware configuration. Once the business provides you 
h the actual volume of transactions, you should design all the performance testing 





scenarios to size your platform correctly and based on this definition. 
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Note Keep in mind that business transactions might not be equivalent to the 
number of messages that BizTalk Server process per business flow, as usually this 
relationship is one:many. 














TL Volume Definition Number of Transactions Per Hour 
9 Extremely high 000,000 

8 Very high (000,000 and < 3,000,000 

7 Hi 100,000 and < 1,000,000 

6 Medium >=100,000 and « 300,000 

5 Common >= 10,000 and < 100,000 

4 low «10000 





Performance Behavior Level (PBL) 
‘This application priority level dictates how an application behaves performance wise. In 
BizTalk Server, applications can be classified int the following categories: 
PBL Mame 
30 Gwen 
B High throughput 
T Mixed 
6 Nospectc 


This is more important than it seems at first glance, as you will have to tune the 
application based on its most common behavior, 


OMPTERA OPTIMIZING THE BIZTALK PATRI 
Service-Level Agreements Between the 
Integrated Parties 


Ase ce provider and a 
consumer. Your BizTalk Server environment provides specific business functionalities 





ice-level agreement (SLA) is a contract between a sen 


that must run below the defined SLA agreement, Although services can hav 





specific custom SLA, for most BizTalk Server solutions your agreement should include 
‘one or more of the following SLAs: 


+ Performance 


+ Availability 


Performance Service Level Agreement 


For BizTalk Server, you define performance SLA at the business application level. The 
concept of an application in integration world is probably quite deceiving as usually, 
BizTalk Server connects to different services to provide the business functionality 
Additionally, mature BizTalk Server deployments have usually shared applications that 
behave as a framework that provides internal functionality to the business applications 
‘or acts as an enterprise service bus. 

As most of the performance settings in BizTalk Server are applied at a host level, it is 
essential that you define a host separation policy that matches the required performance 
SLA. What does it mean exactly? For example, Ifyou are developing an application that 
behav 





sas low latency, the best option is to place all the BizTalk Server artifacts to run. 
ina dedicated set of hosts and apply specific host settings to improve latency. In other 
words, 





you can create three hosts for that application: One for receive locations, another 
‘one for orchestrations, and one more for sending ports. 

In contrast, if the application is running only a few transactions per day, you do not 
need to create additional hosts; you can have a set of standard hosts to run all the low 
volume applications. 

Adding unnecessary hosts can create overhead in the BizTalk and SQL Server boxes 
‘when host instances are started. That is why you have to use common sense when 
approaching a new BizTalk Server project. Do not just create a new set of receiving, 
processing, and sending hosts because it is just the Microsoft best practice. Instead, 
create a generic set of hosts that you will be using for low volume applications, and then 
you move forward with the specific cases. 
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We will discuss the host separation policy in more detail in later sect 
"To create a performance service level agreement, you have to agree with the 
integrated parties in the following elements: 





. A measure oftime— Usually, business decision makers will state that 
the solution must process a specific number of transactions during a 
specifie time range. An example could be this solution must process 
one thousand transactions per minute, per second, or two million 
monthly. 


+ Latency of individual transactions—Sometimes, especially for 
low latency scenarios, individual transactions will also havea 
performance SLA. We define the latency of an individual transactions 
as the elapsed time since the consumer initiated the request and gets 
the response from the integration layer (whatever the format of the 
response is). This, ofcourse, can vary across all of the cases you can 
imagine but usually low latency scenarios will require an individual 
transaction time of fewer than 5-2 seconds (down to milliseconds 
even) Ifyou receive a requirement like this, then you should be 
automatically thought afa low latency scenario and because of that. 
you should consider applying low latency customizations, In some 
other cases, the individual transaction time is not essential. What 
‘matters in these cases, is the total number of transactions globally. In 
other words, a high throughput scenario, 





^ Numberof ransactions-— This is also very important. The number of 
business transactions creates new BizTalk Server messages. However, 
that does not necessary means that BizTalk Server will publish the 
same number of messages as most of BizTalk Server solutions create 
Internal messages to drive the logic through the code. Additionally, 
orchestrations will be consuming internal services, databases and 
“Net components that can publish new messages to the Message Box. 
So, how do we caleulate the number of messages in BizTalk Server? 
‘There is not an easy answer to that. However, the best thing you could 
do asa developer is to document all the messages that your solution. 
is using per message flow. This action will become crucial later for 
testing, sizing, troubleshooting and maintaining the BizTalk Server 
solution, 
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+ Maximum number of live instances —As messages are stored into 
the message box database, itis a must that you define the maximum. 
number of transactions that can be running simultaneously. This 
number is affected by two different aspects: 


+ Consumer requests During the normal operation behav 
‘consumer of the service will send some transactions. Howe 
In the majority of the scenarios, those requests will arrive at. 
BizTalk Server at their pace, and itis very complicated to size 
the environment based on that number. To avoid this problem, 
consumer and provider must agree on the maximum numbe 
of simultaneous transactions that the consumer can request. 
By doing so, you will be sizing the environment based highest 
performance business requirement, 





t, the 








+ BizTalk Server engine behavior—This refers to the transactions 
that are queued in BizTalk Server because ofthe engine features 
such as throttling, pooling intervals, and in-memory and 
processing queues. Getting a specifie value for this is again 
almost impossible. What you could do though is to increment the 
number of maximum requests based on business priority levels: 








BPL The increment of Maximum Transactions 





No increment required 


6 E 
7 E 
8 E 
D Ld 


For instance, imagine that you are designing a BizTalk Server solution that you 
‘categorized as priority 6, The business requirement is that the application must handle a 
maximum of 200 transactions simultaneously. Therefore, if we applied the formula, you 
will have to add 20% on top of the business requirement. That means that BizTalk Server 
should be able to process 40 additional transactions caused by the engine behavior, 
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Convoy pattems—Achieving some business functionalities will 
eventually take you to implement convoy patterns. BizTalk Server 
holds convoys, and the associated message instances, in a separated 
set of tables within the message box, Ifthe number of messages in 
convoy is substantial (especially if they are using large messages), 
you will have tø size the Message Box accordingly to include the extra 
space requirements. Additionally, convoy patterns can cause zombie 
messages within the Message Box database. 








Recovery behavior—How would affect an error to the business? 
In case ofan error, will the caller system be able to resubmit the 
transaction again? What about the destination system? Can it 
handle duplicated transactions by discarding them? For obvious 
reasons this is very important for the solution design but also for 
performance. Why? BizTalk Server has a great feature to recover 
individual orchestrations from the latest service call in case of 

a failure. When an orchestration is sending a request to service 
or, in general, when itis waiting for something to happen (the 
orchestration is idle), the archestration engine will dehydrate that 
orchestration into the Message Box. That means that will store th 
orchestration state into the database waiting for the timed event to 
occur This is very useful too, first it saves hardware resources, and 
second, it allows the engine to recover the orchestration in case the 
failure. If the recovery behavior of the calling system is set to resubmit 
the transaction again, then you do not need the dehydration BizTalk 
Server feature, and you can disable it. Thisaction alleviates SQL 
Server resources as orchestrations will not perform extra round trips 
to the message box. However, keep in mind that the BizTalk Server 
host instance th 
orchestrations will not be saved to the Message Box. 








ins orchestrations will consume more memory as 
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+ Monitoring capabilities—Do the integrated parties need 
the data processed by BizTalk Server? If yes, would it be 
saving the standard BizTalk Server tracking information or do they 
require business data associated with the process as well? Proactively 
‘monitoring performance data should be always considered. 
Predicting when the system can potentially reach a performance 
bottleneck will help you to create a healthy platform and to avoid 
majority of performance problems related to bottlenecks on both, 
BizTalk and SQL Server side. In Chapter 8, you will learn how to 
‘monitor the BizTalk Server platform using BizTalk 360 tool. 








+ Organization or business data retention policies Some 
organizations, or even individual entities within the organization, will 
have specific regulations to save tracking data during a period that is 
far beyond the limit that should be used for troubleshooting. In this 
situation, you might need to keep tracking data for more extended 
periods of time. In this case, the Tracking database can become a 
performance bottleneck aver time. Therefore, you should be ready to 
tune the SQL Server platform, so it has enough hardware resources 
to deal with all the tracking feature. Unfortunately, at the moment 
of writing this book, the configuration ofthe tracking purging job 
applies globally to rhe whole BizTalk group and does not purge 
specific application tracking data, 


Out of-the-box, BizTalk Server can tune a specific host to improve its behavior within 
the following areas: 


^o Message publication 
^o Message delivery 

+ Polling the Message Box database 
^o Dehydration behavior 


‘These topics are discussed later in this chapter and in Chapter 6. 
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Availability SLA 


In information technology, high availability refers to a system or component that is 
uninterruptedly operational for an appropriate length of time. Availability can be 
measured relative to "100% operational" or “never falling” A widely-held but dificult-t0- 
achieve standard of availability for a system or produet is known as “ive 98” (99,999) 
availability. While from infrastructure point of view you can provide the majority of the 
necessary considerations to provide high availability elopment stage 
Where developers can use techniques to get the availability level closer to 99,999%. In 
Chapter 7, you learn about these techniques, 

‘This book does not focus on high availability, therefore, the recommendations in this 
section should be taken as inidal steps. 

Within application priority levels, you must agree with the integration layers and the 
high availability level. The following aspects should come up in this discussion: 














‘+ Recovery downtime—In case ofa significant failure, do the business 
accepts downtime for recovering? 

+ Troubleshooting SLA— What is the maximum downtime of the 
service in case of troubleshooting? 

+ Deployment downtime—To fix bugs and to provide new 
functionality, deployments might be quite frequent. This is very 
dependent on the release stage level ofthe solution. Ifi is the case, 
what is the maximum amount of time that the business accepts for 
deployment situations? 





+ Adapter considerations Depending on the adapters used to 
integrate the systems involved, you will have to consider different 
strategies to provide high availabilty. 





"The following elements in BizTalk Server are crucial regarding high availability 
because ifany of them fails, the system will not be able to process messages, in other 
‘words, these elements are single point of failures in any BizTalk Server environment. 





^ Hosts Instances—Host instances initiate most of the activities related 
to BizTalk Server engine 


+ The End Point Manager—Hosts in-process and isolated 
messaging (but not messages related to orchestrations), 


+ TDS service, which implements the tracking and BAM features 
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+ Although message publication and dequeue processes are 
implemented with SQL Server stored procedures, host instances 
are Initiating the process. 


+ Orchestrations—Ithosts instances are 





ning orchestrations, 
a sub service called XLang engine loads into the host instance 
‘windows service to provide all the orchestration functionality 


“Therefore, you will need to provide high availability for the 
host instances because ifany of those elements are affected, 
the BizTalk engine will not be able to process messages. As 
host instances are windows services, the only way to provide 
redundancy is by deploying at least wo BizTalk Servers, 


+ Enterprise single sign-on service —This service is used in BizTalk 
Server for the following reasons: 


+ Tosave adapter and ports configuration 
+ Tostore custom application configuration data. 
+ To mapa Windows user ID to non-Windows user credentials. 


‘This data is stored in the Single Enterprise sign-on a database thats encrypted in 
the moment of the BizTalk Server configuration stage. Within the whole BizTalk Server 
group, one of the Single Enterprise sing-on services is marked as the master secret server, 
"This is the only one that has the encryption key in memory. The rest of the enterprise 
single sign-on services need to ask fo the key to the master. 

the master secret server becomes unavailable, BizTalk Server will stop processing 
messages, and you will need to set the master secret server to another server manually 
"Therefore, you will also need to provide high availability to the master secret server, 

‘+ Microsoft distributed transaction coordinator (MS DTC)—provi 


the functionality to ensure complete transactions across a distributed 
environment. If the distributed transaction coordinators become 





unavailable, BizTalk Server will fail to enlist a new transaction and. 
it will top processing messages. Later on, we will ook at MSTDC in 
more detail. 
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+ BizTalk Server databases — BizTalk Server stores a significant amount 
ofinformation in the databases. Ifany of the core BizTalk Serv 
databases becomes unavailable, the platform will be able to process 
service instances. Therefore, you should provide high availability for 
the databases using one of the following methods: 





+ Implementing a Microsoft cluster in the SQL Server 

+ Implementing SQL Server Always-On and high availability 
groups 

+ BizTalk Server maintenance jobs—When you configure BizTalk 
Server, the process will create a set of SQL Server agent jobs that are 
extremely important for the platform as they perform operations 
such as backing up databases, maintaining integrity within the 

Message Box and tracking, moving tracking data from the message 

and purging the Tracking database. 





box to the Tracking databas 
‘Therefore, you will need also to provide high availability to the SQL 
Server agent service. 





Designing a high availability will nat cover all the scenarios as there are factors such as 
the human mistakes and the lack of knowledge, that can cause a production environment 
10 become unavailable. For that reason, you should also consider the following important 
aspects that will help you to provide an efficient high availability level: 





+ Application instrumentation—Covered in Chapter 5. 


* Sid 





ide versioning —Covered in Chapter 7. 


+ Proactive monitoring—Cavered in Chapter 8. 


Factors That Are Important for BizTalk Performance 


"The number of factors that affect BizTalk Server performance is very large. In this section. 
we will review the most important ones, as detailing every specific performance setting 
will take probably more than 500 pages, and most likely you are not going to face all the 


situations atthe same time. 
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You will learn about optimizations related to the following areas: 
^ Tracking 
^o Hostseparation policy 
^o Host performance settings 
^o Message size 
^o Message Box database 
. Microsoft distributed transaction coordinator 
+ Windows communication foundation throttling 


+ Concurrent HTTP connections 





Later, in Chapter 6 you learn about how to improve your developments, so the 
development and administration team can work together to boost the BizTalk Server 


platform. 


Message Size 


The message size, along with the number of transactions, have an Impact in th 
following areas: 





. Size ofthe Message Box database— BizTalk Server stores all live 
messages in the Message Box database and uses a set of queue tables to 
hold references to those messages. As message size increase, Message 
Box database increases in size along with the number oflive Instances 

+ Size of the Tracking database—If message body tracking is enabled. 
at any level, Tracking database grows exponentially along with the 
message size, 


+ Pipeline CPU consumption—Pipelines use CPU resources to 
loop through the streamed binary data, therefore as message size 
increases, the pipeline needs to stream more information, and that 
increases CPU utilization. 


+ Map execution—When extracting or setting values with a map. 
the document is loaded into memory and therefore memory 
consumption Increases. 
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+ Orchestration dehydrated size in the Message Box database—Every 
single time an orchestration dehydrates, it saves all the messages 
that are part of the actual scope (messages that are not yet used will 
not be persisted). Therefore, the size of Message Box database is also 
affected because of he number of dehydrated orchestrations. 


+ Orchestration memory consumption—As all orchestration messages 
are stored into the memory, the message size has a direct impact 
on the server memory consumption. This is especially true ifthe 
developer is using XPath expressions or XH Document classes because 
these objects load the message into the memory. 

+ Orchestration CPU consumption—If the process is looping through 
a very large message, the host instance is acting as a handler for that 
orchestration and uses more threads per CPU. 





+ Overall throughput can be affected ifthe size of received message 
increase above the initially defined thresholds. 

^ XPath and XMLdocument classes—As these objects load the whole 
message into the memory, memory consumption increases along 
with the size ofthe message when XPath or XMLDocument classes 
are used, 





+ Number SQL Server locks—The larger the message, the more 
significant the number fragments that the engine uses to save the 
binary data into the Message Box. Each fragment creates one or mare 
SQL Server locks within the Message Box database. Ifthe number of. 
messages is quite large, you can experiment with an out-o-lack error 
In SQLServer. 





Note Keep in mind that if the application receives or sends JSON and fiat file 
messages if you want to use that message for operations such as validation, 
property promotion, and transformation services, they must be transformed into 
XML (using a pipeline). The conversion itself can increase the size of the message 
up to ten times the original size, since XML representation requires element names 
specification. 
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While BizTalk Server does not impose any constraint on message size, practical 
hardware limits might require that you design the BizTalk Server solution in a completely 
different way to reduce message sizes, because large messages require more processing 
resources. However, at the group level, you can control how BizTalk Server processes 
large messages within the Message Box database. 


What Is the Large Message Size Setting? 


the size ofan incoming message is larger than the number of Kilobytes specified for 
this setting, the message is split into fragments of that size. I'size is not exceeded, then 
the entire message is committed as one part. In BizTalk Server 2016, the default value is 
100 KB, 

Lot's see this setting in action using an example: 

Imagine that BizTalk receives a file of 1089 KB. The engine evaluates the message 
data size against the large message size setting In this case, since the setting is set to 











100 KB, 11 fragments are inserted in the Fragnents table: 
^ Ten fragments of 100 KB size 


+ One fragment of 89 KB size 








Warning Changing the large message size could have an adverse/positive 
impact on the environment and the platform could suffer from one ofthe following 
issues: 


DTC Locks Depending on the load, SOL Server could potentially get out of 
transactions causing DTC Locks and will timeout message publication. If you face 
this situation, it might be reasonable to increase the size of the large message, as 
the number of locks decrease exponentially due to fewer fragments inserted in the 
tables. 

Out of Memory On the other hand, increasing this setting could also raise the 
amount of memory consumed by BizTalk and SQL Servers, as fragments will be 
larger. In this case, you should do the opposite: decrease the size until the out-of- 
memory condition disappears. 
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"This setting is configurable using the BizTalk Administration Console, under the 
BizTalk group settings, as shown in Figure 4-2. 








— —Ó— 
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Figure 4-2. Large message size setting 


Asyoucan see there is not a value thar fits all scenarios. The recommendation isto start 
‘with the default one (100 KD) nd, f you experiment any of those sues, you could leverage 
‘onthe recommendation ofan exellent BizTalk Server too the health monitor (BHM). 

‘To check the recommendation value for the Large Message Size setting, once BHM 
tool has generated your report you should navigate to section BT DBs Details and from 
there locate the element BizTalk - All MsgBox Dbs: Large Message Fragment Size Tuning 
See Figure 4-3. 
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Figure 4-3. Location of the Large Message element report. 
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Atter you click on the element, you will see a report like the one in Figure 4-4. 
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Figure 4-4. Location of the Large Message element report 
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Note To retrieve this information, the BHM tool queries the Tracking database. 
Therefore, if you want to rely on this recommendation, message body tracking 
should be enabled while the tool is performing the analysis. As you will leam later, 
enabling message body tracking is recommended punctually over the following 
scenarios: 

1. Troubleshooting issues related to message content, 

2. Executing BHM with the objective of getting a large message size 
recommendation. Ideally, you should do this in a no-production environment by 
simulating load in a testing server 

When you finish with either of these tasks, you should disable message body 

tracking again (of course, only if data retention policies allow to do that). 








The most important field on the report is the Suggested Large Message Si 
asit gives you the most accurate recommendation based on the message sizes that ar 
currently published in your environment. In this case, the tool is recommending using 
the default setting of 100 KB, 





Tracking 


"The general best practice for tracking is to not to enable iton production. Howe 
inn 
especially rue for newly released applications, as they can ralse exceptions that 








world scenarios, tracking becomes essential when troubleshooting, This is 
were not considered during the design and development stages. The most important 
considerations for tracking are: 

+ Thesize ofthe message 

^ The tracking level you want for your BizTalk Server artifacts 


+ Promoted properties 
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Additionally, the tracking feature involves a three-step process: 


L. Tracked data will be first placed on the tracking data tables into 
the Message Box database. 


2, The Trackedhessages Copy BizTalksgBoxDb job moves 
message body data to the Tracking database and it runs once per 
minute by default 





3, The DTA Purge and Archive job will delete old tracking data based 
on the job configuration. For instance, ifyou have set the job to keep 
seven days of data, this is the amount of maximum tracking data that 
the tracking database will keep for that specifie business process 


Therefore, there are two important considerations to size tracking properly: 


+ Amount of tracking data per minute—As the TrackedMessages_Copy_ 
BizTalkMsgloxDb runs every minute, the Message Box database will 
keep tracking data for one minute also. This is the reason; tracking 
can become a bottleneck that can affect the BizTalk Server runtime. 
Iis crucial that you consider this factor when designing the tracking 
settings for a specifie solution. Therefore, you must calculate the 
size of the Message Box database based on the message size and the 
number: 





instances per minute 
. Amount oftracking data thar the purging job keeps—Ifyou set 
the purging job to keep seven days af data, you should calculate 
the size of the Tracking database based on this number, the 
message size, tracking data points, and the number ofinstances 
for those seven days, 
Tracking Levels 
Tracking can be customized at the following BizTalk Server artifacts: 
Receive locations 
+ Orchesrations 
. Sending ports 


+ Schemas 
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+ Pipeline 


^o Business rules. 


Orchestrations 
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At the orchestration level you have the following tracking options: 





Tracking Point Observations 





Events- This wil track when the 

Orchestration orchestration starts and 

starvend ends. 

Events-Message This wil rack an 

senureceve event every time an 
orchestration receives or 
sends a message. 

Events Shape Tracks when a 

startend orchestrations shape 
starts and ends. 


Track message Saves the message 
bodies before! bodies before a and after 
after orchestration processing. 


Track message Will track promoted 
properties properties for incoming 
and outgoing messages. 


it depends on the number of instances. 


"t depends on the number of instances. 


"t depends on the number of instances. 


This is the most expensive configuration. Whenever 
possible try to avoid it and ìf you have to save the 
content of the message, its better that you enable 
itat receive or send port only. By doing so, you will 
have more control because you wil save specific 
messages and not all the messages processed by 
the orchestration. This setting is only useful when 
you want to track a message that s not processed 
by any port, 

‘The same principe as the track message body 
setting, but related to promoted properties. 
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Receive Ports 


"The following tracking options are available at the receive port level 





Tracking Point Observations Impact 





Track message Saves the message — Thisis the most expensive configuration. Whenever 
bodies before/after bodies before and possible, try to avoid it. 
port processing after port processing, 


‘rack message _Willrack promoted The same principle as the track message body setting, 





properties properties before and- but related to promoted properties. For the majority ot 

before/after port ater port processing scenarios, you wil not need to track properties on the 

processing incoming side. Oniy when you want o troubleshoot 
Based on a promoted property in the incoming 
message. 

Sending Ports 


‘The following racking options are available at the receive port level: 





Tracking Point Observations Impact 





Track message Saves the message This is the most expensive configuration. Whenever 
bodies before/after bodies before and after possible, try to avoid it. 
port processing port processing. 


Trackmessage Wil wack promoted The same principle as the rack message body 


properties properties before and setting, but related to promoted properties. For the 

beforeafter port after port processing majority of scenarios, you will not need to track 

processing properties in the sending side. Only when you want 
"o troubleshoot based on a promoted property in the 
sending message 
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For schemas, there is only one tracking configuration setting 





Tracking Point Observations 


Impact 





‘Always track al Wil track promoted 
properties properties everytime a 
message based on this 


It you need ever to track properties, it should be 
explicitly done at orchestration or port level. This 
‘should be used oniy in the development stage 


schema is sent or received. where you want to troubleshoot and find bugs in 


early stages of the project. 








Pipelines 


At the pipelines level you have the following tracking options: 





Tracking Point Observations 





Events-portstart This wil track 


and end when the pipeline 
starts and ends 

Events-Message This will rack an 

sentfeceive event every time an 


Pipeline receives or 

sends a message. 
Track message Saves the message 
bodies before) bodies before a 
after and atter pipeline 

processing. 


1t depends on the number of instances. 


It depends on the number of instances. 


This is the most expensive configuration. Whenever 
possible, try to avoid it and if you must save the 

content of the message, its better that you enable it 

at receive or send port only. By doing so, you will have 
more control because you will save specific messages 
and not all the messages processed by the pipeline. 
Additonally, pipelines can be used in several ports 
therefore, i you want to enable a specific pipeline to 
"rack information, ts better that yau set tracking only at 
the port level. 
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Assigning Tracking Configuration Based on the 
Release Stage (RS) Level 


the business does not require a heavy data retention policy 
guidelines to enable tracking: 





you use the following. 





RSL Level Description 





9 — Fitstrelease Message body tracking at sensible points can be enabled. Orchestration 
events must be enabled to allow developers to troubleshoot 
orchestration issues. AL this stage, you can enable also tracking for 
promoted properties if that is used for troubleshooting. Al in all, tacking 
an be enabled in all areas. 

3 Unstable release Applications is now performing for a while but there are stil some bugs 
that are not fied yet. Partial tracking is enabled to help developers 
fixing issues. 

7 Unstable release All known bugs are fixed. Tracking is enabled at orchestration level 

(performance) to find performance bottlenecks anf. No message body tracking is 
enabled at this stage. 

6  Stablerelease The application is now stable and running without known issues for & 
while now. Tracking is disabled at all artifact levels, and standard host 
setigs are applied 





As discussed previously, tracking is a feature that uses many resources of BizTalk and 
SQL Servers, thus you should Isolate the tracking functionality to run in a dedicated 

host, so that the tracking functionality does not need to compete with the rest of BizTalk 
Sen 





ate a Host for Tracking 








+ functionalities (receiving, processing, and sending). 
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High availability is also essential because ifthe host instance performing the tracking 
functionality fails, the following areas of BizTalk Server will be also affected: 


+ Tracked data will not be moved from the Message Box to the Tracking 
database, 


. BAM tracking data will not be moved to the BAM primary import 
database, so all your activities will not reflect current data, There isan 
exception for this situation where developers write BAM data directly 
to the BANPrinaryInport database using the DizectEventBuffer 
API, and that data is not stored previously in the Tracking database. 


As you can see, both situations can cause the Message Box to grow aver time, and. 
that could initiate a bottleneck to the message publication and the dequeue processes. 
Therefore, the tracking host should be running in at least wo servers by creating two 
host instances. 

‘When adding more Message Boxes (see the section Message Box database later 
in this chapter), the recommendation is to have the same number of tracking host. 
instances plus one more, for high availability reasons. Imagine that you have two 
Message Boxes, so the total number should be three. Ir you do not add more tracking 
host instances, you will have only one that has to move data from the three Message 
Boxes and, depending on the tracking settings, that could also cause a battleneck on the 
Message Box, as the tracking data could not be moved efficiently to the tracking DTA and 
BAM databases, 





Purging the Tracking Database 


As BizTalk Servers processing messages, Tracking database grows along with the load (if 
tracking is enabled for the BizTalk Server artifacts involved in the process) and the maximum 
sustainable load can be affected, especially ifthe Tracking and Message Box databases are 
placed on the same SQL Server instance or If the database fles are on the same disk. 

The tracking feature is very transactional, and it performs many modification queries 
to the tracking data files, Thus, the transactional data and log fles for the Tracking 
database can grow very quickly. This is not a good thing from SQL Server performance 
point of view. The larger the ile, the greater the chances of entering into a disk 
contention issue (along with a high number of write and read operations). 
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When you configure BizTalk Server for the Bist time, a SQL Server job called DTA 
Purge and Archive (Bi2Tal TAD) is created. This job is used to purge the BizTalk 
Server database, and as it requires that the user configure all the settings. Itis disabled by 
default. 

"This job performs the following tasks: 





+ Purges the Tracking database following specific crit 





. Itean archive the deleted data to an archiving server. 
"To delete all the data, the job can call on the following stored procedures: 


+ dtasp_BackUpAndPurgeTrackingDatabase: It performs a backup of 
the data that will be purged, 





«^ GnHours—All completed instances data, older than the number of 
hours, will be purged, 

‘+ GnDays—all completed instances data, older than the number of 
days, will be purged. 


‘+ QnHardDays—All data (completed or not) older than the 
number of nHardDays days, will be purged if 
@fhardDeleteRunningInstances is set o 1. 


+ GnvcFolder—Eolder where you want to create the backup files, 


+ GoveValidatingServer—Server name that will be used to archive 
the purged data. This server should be added to the linked 
servers ofthe SQL Server instance holding the Tracking database. 








+ @flardDeleteRunningInstances—Irthis parameter is set to 1, 
the stored procedure will delete all the running service instances 
older than hard delete days, 


+ dtasp_PurgeTrackingDatabase—Purges the database without 
performing a backup frst, Performance wise, this is the most efficient 
‘option as depending on the tracking size, the backup operation, 
can take a long time. if you do not have data retention policies, you 
should use this option whenever possible 
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+ Parameters explanation: 


+ @nHours—All completed instances data, older than the number of 
hours, will be purged, 


+ @ndays—All completed instances data, older than the number of 
days, will be purged. 

+ @nHardDays—All data (completed or not) older than the number 
of@nHardDays days, will be purged. 

+ @dtLastBackup—The stored procedure uses this date to ensure 
that the process does not delete data that was not in the last 
backup. Ifyou pass through the date and time value of the 
‘moment of running the job (using getUTCDate( )) the stared 
procedure will delete all the corresponding data. This parameter. 
is used internally to make sure the date of the execution is older 
than the date calculated by niazdDays, nDays, and nHours. 








+ @flardDeletekunningInstances—If this parameter is set to 
the stored procedure will delete all the running service insta 
older than hard delete days. 








Important The parameter @#HardDeleteRunningInstances for both stored 
procedures is not exposed by the SQL Server job default call definition, You must 
manually add it to the call and set it up to 1, as the default value is O. If you do 
change it the stored procedure will not delete all the running instances that are 
older than the @nHardDays parameter. 





Editing the job using the SQL Server job interface s usually not clear, as the editing. 
window does not provide InteliSense, What you can do, though, is to copy the default 

using SQL Server management studio, paste itin a new 
SQL Server query. Figures 4-5 and 4-6 show you the difference, 








job step configuration and ther 
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Figure 4-5, Viewing the job configuration using the job properties 





The same code is shown in the SQL Server Management Studio Query 
Figure 46, 
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Figure4-6. Viewing the job configuration using the Query Editor 


Host Architecture 


‘As you learned in Chapter 1, host instances are implemented as Windows services. 
These processes run on the server using their own set of hardware resources such as 
memory, threads (CPU), and 1/0 (networking and disk). 
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By default, the BizTalk Server configuration creates only two hosts: The 
BizTalkServerāpplication, which is used as an in-process host, and the 
BizTalkServerlsolatediost, which controls all the requests received by external 
processes (IS in the majority of scenarios of today). With this configuration, you have all 
that you need o start developing and deploying BizTalk Server solutions to production. 
However, the initial configuration is not optimum for a real production scenario, as all 
BizTalk processing artifacts such as receive locations, orchestrations, and send ports will 
be using the same in-process host configuration (BizTalkServertpplication) and that 
will eventually cause queuing situations into the Message Box database. 

For instance, imagine that you have a solution that receives 1,000 large messages. 

In this scenario, message publication process uses all the available memory and CPU 
usage before the default throng condition arises. I the host instance is busy at message 
publication stage, neither processing nor delivering functionaites will take place, because 
the hoast Instance does not have resources to start both operations. The same applies to 
tracking funetionaliy; ifthe host instance is busy performing the rest of the operations 
‘might not have threads, memory or 1O to allocate resources and perform the tracking 
operation (inserting tracking messages to the Message Box and Tracking databases). 

Ifyou decide, however, to separate hosts based on BizTalk Server functionality, even 
though the publishing host enters in thronling condition, your orchestrations and send 
ports will still send messages, as they will not be affected by the same throttling condition 

Additionally, as most ofthe performance settings in BizTalk server are applied 
ata host level, it is essential that you define a host separation policy that matches 
the required performance SLA. What does it mean exactly? For example, ifyou are 
developing an application that behaves as low latency (performance behavior), the 
‘best option is to place all the BizTalk Server artifacts to run in a dedicated set of hosts. In 
‘other words, you can create three hosts for that application: One for receive locations, 
another one for orchestrations, and one more for sending ports. 

‘On the other hand, ifthe application is running only a few transactions per day, you 
do not need to create additional hosts; you can have a set o standard hosts to run all the 
low volume applications. 

Keep in mind that adding unnecessary hosts can create overhead in the BizTalk and 
SQL Servers when host instances are started. That is why you have to use common sense 
when approaching a new BizTalk Server project. Do not just create a new setof receiving, 
processing, and sending hosts because itis the Microsoft best practice. Instead, create 
a generic set of hosts that will be used for low volume applications, and then you move 
forward with the specific eases. 
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‘As you can see, choosing the right host separation policy is not easy, Without a 
doubt, itis one of the most relevant decisions that you will be taking. A good starting 
point is tø les 

Tn the following sections, you learn about an optimal initial configuration 





rage on the application priority levels and start from there. 


and general recommendations based on the different application priority levels. 
The recommendations detailed in the next sections should not be taken as Dat 


recommendations because testing will confirm the best optimum configuration. 





Initial Host Separation Policy 


‘As you learned in the previous section, the default BizTalk Server host separation policy 
is not optimal for the majority of scenarios. You can use the following table asa starting. 
dein 





point, Further customization can be applied based on the application priority lex 
‘combination with performance testing, 





Host Name Description 





Tracking  Pertorms the tracking functionality. Also, itis the only host with tracking enabled. 
It should be 64-bit. 

Receiving32 This host is designed to run all the receive locations that handie adapters that 
only work in a 32-bit mode, such as FTP, POPS, and the old SQL Server adapter 
(deprecated 

Receiving64 ^ This hostis designed to run al of the receive locations that handle adapters that 
can work in a 64-bit modes, such as Fle, HTTP. MSMO, MOSeries, SFTP SMTP. 
‘SOAP and WCF. 

Sending32 This host is designed to run all of the receive locations that handle adapters that 
only work in a 32-bit mode, such as FTP, POPS, and non-WCF SOL Server adapter 
(deprecated 

Sendingéà This host is designed to run all of the receive locations that handle adapters that 
can work in a 64-bit mode, such as File, HTTP. MSMO, MOSeries, SFTP. SMTP, 
SOAP, and WCF. 

Processingt2 — This host will un orchestrations that for any reason has 32-bit requirements. This 
is usually required when a custom NET component can run in 32-bit mode only. 

Processing64 The rest of the non 32-bit orchestrations should run under this hosts, 
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Note Ifthe BizTalk Server group does not require 32-bit adapters, you should 
not create 32-bit hosts. Also, you should set the Show Performance Counters to 
orchestrations if the host is dedicated to orchestrations. 





Host Separation Policy Based on Application Priority Levels 


In this section you will learn how to use the application priority levels to design an 
optimal host separation policy. 


Host Separation Guidance for Release Stage Level (RSL) 


"The release stage level reflects how mature is an application from the issues point 
juse of that, it may be interesting to Isolate applications that are not y 





of view. Bec 








stable. You can follow the following suggested recommendations based on the 


stage levels: 





RSL Level Suggested Recommendation 





aB Firstorearly The application should have its own set of hosts due o the number ot 
releases deployments required to fi the upcoming issues. By doing so, the rest ot 
the applications wil not be affected by deployments, as in the majority of 
the scenarios you will have to restart host instances or terminate running 
instances (f no side-by-side version is enabled, of course). 

7 Unstable release At this stage, aplication functionality is granted. However, performance 

(performance) i stil an issue, and new deployments can occur frequent, Additionally. 
hosts might have temporary settings to overcome the performance 
issues caused by inappropriate development decisions. 

6 Sable elease The application is now stable and running without known issues for a 
‘while now Tracking is disabled at all artifact levels, and standard host 
settings are applied. RSL should not be considered as a reason to isolate 
this application into a different set f hosts. 
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Host Separation Guidance for Business Priority Level (BPL) 


"The business priority level reflects the application importance from the business point of 
View. You can use the following suggested recommendations to apply a host separation. 
policy that matches the business priority level 





BPL Suggested Recommendation 





9-7 Applications must have thelr own set of dedicated hosts for receiving, processing, and 
sending functionalities. 

6 As business accepts downtime, applications with the same BPL (6) can be assigned to the 
same set of hosts f they share similar performance requirements. Creating a set of hosts 
per BizTalk functionality might be required it performance for one of the applications in the 
BPL level is significant, but the application is not considered mission-critical. 

5 Applications should be placed in only one host to handie all operations, but this host cannot 
be shared with the rest of the applications (no host separation based on BizTalk Server 
functionality is required). 

4 Applications can be running in a shared unique host (no host separation based on BizTalk 
Server functionality is required). 





Host Separation Guidance for Transaction Levels (TL) 


The 


You can follow these recommendations for an optimal host separation policy based on 





nsaction level shows the number of transactions that the application process, 





this level: 





TL Volume Number of transactions Host Separation Recommended 
definition Per Hour 











9 Extremely >=3,000,000 Yes 
high 
a Very High 000,000 and< Yes 
3,000,000, 
7 High Yes 








(continued) 
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TL Volume Number of transactions Host Separation Recommended 
definition Per Hour 








00,000 and < It depends. You can put in the same set of hosts. 
300000 applications that have similar TL requirements. 


6 Common — »-10,00020d < 100,000 No 
7 tow «10000 No 








Host Separation Guidance for Performance Behavior Level (PBL) 


"The performance behavior level reflects the normal behavior ofthe application 
regarding latency and throughput. To apply the right host separation policy, you can. 
follow these recommendations. 





PBL Description Suggested Recommendation 





3 Lowlatency Yes 
B High By defaut, hosts are configured for high throughput This should suit the 
‘throughput — majority of the scenarios, but if the application needs unique host settings, 
then you need to separate hosts per functionality also. 
7 Miel These are the most challenging one as onl testing will give you the right 
configuration. Use common sense: f you cannot reach your performance SLA 
using the default settings, then you need to split hosts by functionality, 





Host Performance Settings 


Especially when Lam delivering BizTalk Server performance review services, frequently 
hear the following question: How can I give priority to a specific application or e 





anartifact within the application? 
BizTalk Server has only one setting to control priority at the logical l 
been reviewed in Chapter 1, priority at the sending port level. Ifyou adjust this setting, 


ditas 





it guarantees that the dequeue process for that send port puts those instances on top of 
the queue, but not for all ofthe instances. Even though this feature is there, it does not 

ensure priority under peak, or high load, as the BizTalk Server engine can eventually rise 
throttling conditions that can 





mpact priority. 
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However, it is an excellent practice to decrease priority for all the send ports ofthe. 
applications that rank higher in the following application priority levels: 


+Business priority level (BPL) 
+ Transaction levels (TL) 
‘+ Performance behavior level (PBL) 


Keep in mind that if you change priority on all the send ports (ar the majority of them), 
the Improvement becomes closer to nane because all send ports will have the same 
priority. Remember that you can adjust this setting to a value that ranges from 1 to 10. 

Tn BizTalk Server, you give priority to applications by allowing them to use more 
hardware resources. This i a very efficient way to assign priority because, if the host 
instance takes more “dedicated” resources it will perform faster than the rest of th 
‘host instances as it will not be competing constantly for those resources. Additionally, 
when the high priority application runs under a high volume of data, it will throtle with 
specific throttling thresholds that will guarantee its performance SLA, and the rest of 
applications can still use the common amount of resources without getting impacted. 
"This is another reason that the host separation policy becomes extremely important. 

‘Thanks to the flexibility of the BizTalk Server configuration layer, you can take 
the host separation policy as far as to dedicate servers to run specifie functionalities 
(receiving, processing, and sending) or even specific orchestrations or ports. 

BizTalk Server allows you to change the host performance behavior fr the following 
categories: 





+ General host settings 


+ Resource-based throttling. These settings have been explained in 
Chapter? 


+ Rate-based throttling These settings have been explained in Chapter 2. 
+ Orchestration throttling. 


The recommendations in this section should be considered as initial adjustments 
only. At the end ofthe day, only performance testing will drive you tothe right 
‘configuration settings and hardware sizing 
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General Host Settings 


In this section you will learn about the general host settings and how you can adjust 
them based on the performance behavior level (PBL). This level reflects the normal 
behavior ofthe application in terms of latency and throughput. 

You can access the host performance settings using the BizTalk Administration 
Console and selecting a specific host, as shown in Figure 4-7, 
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Figure 4-7. Accessing the host settings 


Move Tracking Data to DTADB 


"This setting enables the host to move data from the Message Box to the Tracking. 
database. Only the host that is dedicated to tracking functionality should have this 
setting enabled. If the host is running artifacts that provide application functionality 
such as orchestrations, the receive and send ports it must be disabled. 
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32-Bit Only 
This setting regulates whether the host is running asa 32-bit or 64-bit process, The only 
situations where you wil be adjusting this seting to 32-hit are: 
+ 32-bit hardware or 32-bit version of Windows. 
«o Some adapters, such as FTP, POPS, and the old SQL Server adapter 
are not supported to run in the 64-bit. 
«o When you are consuminga custom NET component that only works 
With 22-bit Remember that you can consume NET components in 
the following areas: 
+ Orchestrations 
«o Maps 
+ Pipelines 
Assuming that hardware and operating system are 64 bits, you should always disable 
this setting for the host that is doing the tacking functionality, as t must run in the 64 
version. 


Allow Multiple Responses 


By default, host instances cannot process more than one subscription for a response 
message that is being recelved through a request/response port. When the engine 
receives multiple messages, you will get the following error: 


Error details: The message found multiple request-response subscriptions. 
A message can only be routed to a single request-response subscription 


Enable this setting if you want to enable receiving multiple responses in a two-way 
port scenario. 

Response Timeout in Minutes 

"This is the default timeout for all the ports ofthis host. Developers can overwrite this 
setting by code in dynamie ports or by setting the properties at the adapter level in 
the port configuration. The default configuration is set to 20 minutes, which in most 


scenarios is very high. As this setting is per host, consider adjusting this setting to more 
suitable value based on the maximum response time for the consumed service, 
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Maximum Engine Threads 


"This setting controls the maximum number of threads per CPU that the host instances of 
that host will allocate to process messages. The engine load threads based on the current 
load of messages, up to the maximum engine threads setting. 





Note Donat confuse this setting with the NET CLR settings at the host instance level, 
as they control how host instances access CPU resources to perform I/O operations, 





For instance, imagine that your server has 16 cores. If you leave the default value, the 
ite a maximum of 20 * 16 (320) threads for that specific host. 

Changing this setting can h 
can follow this advice: 


message. 





ngine will allo 








ica negative or positive impact on performance. You 





Situation Suggested Recommendation 





Message Box database servers show high CPU Decrease if scaling SOL Servers is not an option. 
utilization, and itis affecting BizTalk Server 

performance by increasing latency. Usually, SQL. 

Lock times are higher than 500 miliseconds. 

‘SOL Server holding Message Box and BizTalk ^ Increase if he system requires more throughput 
Servers are under the CPU utilization stated by or if you want to make more use of CPU 

"he maximum sustainable oad, resources. 





‘Show Performance Counters For 


"This setting controls how BizTalk Server populates the performance counter. 
information. Itis more important than it seems because [vou do not 
performance counters could not be populated with data. 





etit up correctly, 





Value Applicable To 





Messaging Hosts that are not running orchestrations. 
Orchestration Hosts that are dedicated to orchestration processing. 


Active service For hosts that are nunning more than one functionality (receiving, processing, and 
sending). 
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Pooling Intervals 





As you learned in Chapter 1, 
creates a set of queue tables: 
tables to check whether there are new messages associated with that host. While 
messages exist, the interval settings are ignored, and the host instance starts working on 





ssages are stored in the Message Box. As BizTalk Server 
lated to a specific host, host instances will access those 





the pending messages at maximum speed using the available host instance resources, 
When the queue tables are empty, the host instance will access the queues using the 
interval defined by the polling interval setting, 

Ifthe host instances are processing a few service instances a day, you should not 
decrease the pooling interval because that will Increase CPU utilization on SQL Server, 
as most ofthe time the host queue tables are empty. Moreover, you should consider. 


Increasing it. 
Setting Default Value Applicable To 
Messaging 500ms Hosts running pots 


Orchestrations $00ms Host running orchestrations 


‘Suggested Pooling Intervals Based on Application Performance Behavior 


"The performance behavior level reflects the normal behavior ofthe application 
regarding latency and throughput. In order to apply the right host pooling interval 
settings, you can follow these suggested recommendations: 





PBL Description Suggested Recommendation 





9 — Lowlatency Decrease to the point where MST is granted. A good starting point wil be to 
decrease this setting by 50% for every testing cycle. First, you can decrease it 
to 250 and test. I performance is improved and MST is stil granted, then move 
"o decrease 50% more. Repeat testing until MST is exceeded. 

3 High BizTalk Server is shipped for high throughput scenarios. Therefor, the default 

throughput 500 miliseconds should fit the majority of needs. t your application needs to deal 
with a very large number of messages where the load occurs in very specific 
"mes (nat all alang the day), you should test whether increasing this value wil 
have a positive impact. Otherwise, leave the default af 500 miliseconds. 
(continued) 
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PBL Description Suggested Recommendation 





7 Mixed These are the most complicated applications to tune. Only testing vill give you 
"he actual picture, and proactive changes most likely wil nat hl. 

G — Nospecific Leave the default $00 milliseconds if the application is running under the 
performance SLA, 





Let's review these recommendations with a few examples. 


Scenario 1: High Throughput (I) 


A BizTalk Server application receives 1,000 messages through a receive port. Pooling 
interval settings are set to the default values 





+ Messaging interval set to 500. 


. Orchestrations interval set to 500 





1. Thehostinstance is running, and because the load did not start 
yet (there are zero records in the host queue tables) it will poll the 
host queue tables every 500 milliseconds (two times per second) 
to retrieve the new messages when they arrive. 








A receive location publishes 1,000 messages to that host. 


3, The message engine stores those 1,000 messages into the host. 
queue tables of that host after message publication, 

4. Now, because there are 1,000 records in the host queue tabl 
host instance will not pol the tables until dhe host queue tabl 


become empty. In other words, until the host instance processes 
all messages. 





the 





When all queues are empty, if receive locations are not publishing 
more messages, the host instance will start polling based on the 
pooling interval period until new messages arrive. 


This scenario is atypical high throughput scenario where you are receiving a large 
number of messages most likely in a batch scenario. When a batch is received, host 
Instances will process everything without checking ifthere are new m 





sages until the 
end ofthe process, 
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Scenario 2: Low latency 


A BizTalk Server application receives 50 messages per second through a receive port. 
Pooling interval settings are set to these values: 


+ Messaging interval setto 50 
+ Orchestrations interval set to 50 


1, Thehost instance is running, and because the load did not 
start yet (there are zero records in the host queue tables), 
1t will poll the host queue tables every 50 milliseconds 
(twenty times per second) to retrieve the new messages 
when they arrive. 


2. ‘The consumer application starts sending messages one by ane 
‘but ata rate of50 messages per second. You can imagine that 
consumers are retreating money from ATMs and that process 
will work 24x7. 


3. The message engine stores those 50 messages into the host 
queue tables of that host. 





Now, because there are 50 records in the host queue tables, 
the host instance will not poll the tables until the host queue 
tables become empty. In other words, until the host instance 
process all messages. 








‘The consumer sends another 50 messages while the host 
Instance is still working so the queue increases over time, each 
time th 





nsumers request new ATM operations, 


6, Asconsumers can work the whole day, the host queue tables 
might not become empty again, but if they do, host instances. 
will poll them 20 times per second to ensure new instances are 
processed extremely fast. 





the receive locations are not. 





When all queues are empty, 
publishing more messages, the host instance will start polling 
again until new messages arrive. 
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Scenario 3: High Throughput (II) 


You developed a solution that receives one large fle daily at midnight. The file contains a 
collection of transactions that should be processed during the night. The application can. 
receive up to 10,000 transactions each time. 

‘As this application will be initiated only at midnight, you e 





ible scheduling at 
port just to guarantee that even though new files arrive during the day, itis only 
at midnight when the process starts, 

Additionally, you increase the pooling intervals to 200,000 (200 seconds) for the 
hosts dedicated to this application, so that the message engine does not continuously 
polls the Message Box database. 





Conclusions: 


+ Decreasing these values will guarantee that newly published 
messages are processed faster, as host instances will pool the 
database more frequently when the queues are empty. 





Increasing these values, the BizTalk Server engine will not detect 
new messages with the same frequency. Thus, CPU consumption and 
10 on SQL Server will improve when there is no load for those host 
Instances. This is ideal for applications that work punctually during 
the day. 





Note As always, keep in mind that changing these settings is intrinsically linked 
to the type of application and the performance SLA requirements. You should 
always test the changes— especially this one, because it has a significant impact 
on SOL Server CPU utilization and VO. 





Orchestration Throttling 


‘You can control the dehydration behavior of orchestrations by accessing the following 
settings using the BizTalk Administration Console at the host level, as shown in Figure 4- 
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Figure 4-8. Orchestration throttling settings at the host level 
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Dehydration Behavior 


^o Never This option disables the dehydration feature for all the 
orchestrations running under that specifie host. That means 
that when an orchestration is consuming a service or waiting for 
another event to occur, the orchestration engine will keep the 
orchestration into memory. This option will alleviate SQL Server 
CPU, memory, and IO, as the amount of data that SQL Server has 
to process decreases, and networking usage when SQL Server is 
In a separated box. But it will cause the BizTalk Server machine to 
consume a larger amount of memory, and CPU threads might be 
‘busy waiting for an activation event to occur. 
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Always—Dehydration will always occur: While this option will save 
memory resources to the BizTalk machine, it can potentially cause 
an overhead to the SQL Server hosting the Message Box database. 


Custom—Dehydration will take place based on the time based or 
subscriptions thresholds 


^ Time Based 


Maximum threshold—Idle orchestrations will be held into the 
host instance memory, for a maximum number of seconds 
specified by this setting. After that, the orchestration engine 
forces the dehydration of the orchestration (Its still idle). The 
default value is 1800 seconds (30 minutes). 

Minimum threshold—Orchestrations are considered for 
dehydration when they are idle for at least the number of seconds. 
specified in this threshold. The default value is 1 second, 


For example, ifyou set the Minimum Threshold to 1 second, and all the 
orchestrations of that host are completed in less than 1 second, the orchestration engine 
will never dehydrate those orchestrations. On the other hand, if some orchestrations 
instances are idle for more than 1 second, the orchestration engine considers those 
orchestrations for dehydration (depending on the internal algorithm based on elements 
like number of instances running and resource utilization). 

T for some reason, there are orchestration instances taking longer than the maximum. 


threshold, the orchestration engine will force det 





hydration if orchestrations are idle. 


+ Subscriptions. 


Pause AL Orchestrations have subscriptions to messages that 
are stored to the main host queue tables. When the number 
of messages in this queue (associated with the orchestration 
subscription) is equal or higher than the Pause At threshold, the 
messages are not delivered to orchestrations Instances until the 
number of messages in the host queue decreases to the Resume 
At threshold. 
Resume At—As explained in the Pause At setting, this threshold 
controls whether the message engine starts to deliver new 
message instances to orchestrations. 
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For example, ifyou set the Pause At threshold to 1,000, the message engine will stop 
delivering messages to the orchestrations running under that host when the number of 
associated messages to that orchestration reaches 1,000. 

If Resume At is set to 600, the message engine will resume delivering messages to 
those orchestrations when the number of pending messages decreases to 600 or below. 


Automating Host Settings 


"The best way to optimize the BizTalk Server platform, and any other Windows platform, 
is by automating all settings using PowerShell scripts or the out-of-the-box BizTalk 
Server features to Import and export the settings. 

In the previous sections, you learned the most essential BizTalk Server settings. This 
section focuses on automation, and you will learn how to implement automated scripte 
to improve the BizTalk Server and the platform environment. As the majority of the 
techniques detailed in these sections use the PowerShell, this section requires strong 
PowerShell knowledge. 

Since most ofthe settings are applied at the host level, itis crucial that you. 
understand the WMI classes that BizTalk Server exposes. 





What Is WMI? 


Windows Management Instrumentation (WMI) isa collection of classes for management 
data and operations in the Windows world, You can develop your WMLseripts to 
‘automate tasks on servers and to access product functionality. A the moment of writing 
this book, BizTalk Server exposes 31 classes that expose the majority of BizTalk Server 
‘administrative actions. These classes are defined in the following table. 
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Cass 


Description 





MSBTS, Maptersetting 
MSBTS BTSObject 


MSBTS DeploynentService 
MSBTS GroupSetting 


MSBTS Host 
MSETS HostInstance 
MSBTS HostInstancesetting 


MSBTS HostQueue 
MSBTS HostSetting 

MSBTS MessageInstance 
NSBTS_MessageInstanceSuspendedévent 


MSBTS HsgloxSetting 


MSBTS Orchestration 
NSBTS_Receivetiandler 
MSBTS Receivelocation 
NSBTS_ReceiveLocationdrchestration 


WSBTS_ReceivePort 
SBTS_Sendandler 
MSBTS_Sendandler2 


Registers new adapters 


This member supports the BizTalk Server. 
internal intrastructure. You should not be using 
this in your code, as it is not supported. 


Encapsulates BizTalk assemblies for deployment 
‘or undeployment and bindings export or import. 


Represents a BizTalk Server group associated 
with a specific management database 


Represents a BizTalk server host. 
Represents a host instance. 


Used to read and update the host instance 
settings. 


Represents the main queue of a specific host. 
Used to represent the settings for a host. 
Represents a message instance. 


Represents a suspended event for a BizTalk 
message instance. 


Represents a single Message Box setting in the 
BizTalk Server group. 


Represents an instance of an orchestration. 
Represents a receive handler. 
Represents a receive location. 


Represents receive locations that are linked to 
orchestrations. 


Represents a receive port 
Represents a send handler. 
Represents an extended individual send handler. 





(continued) 
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Cass 


Description 





NSBTS_SendPort 
MSBTS, SendPortGroup 

MSBTS, SendPortGroup2SendPort (WMI) 
MSBTS Server 


MSBTS ServerHost 


MSBTS. ServerSetting 


MSBTS Service 


MSBTS. Servicelnstance 


NSBTS_ServiceInstanceSuspendedEvent 


MSBTS Setting 


MSBTS TrackedessageInstance 
MSBTS_TrackedMessageTnstance? (WHI) 


Represents a send port. 
Represents a group of send ports. 

Same as the previous one. 

Represents computers within a group that have 
BizTalk Servers installed. 

Reflects mappings betwen BizTalk Servers and 
BizTalk Hosts. 

Represents specific computers within the some. 
BizTaik group. 

This member supports the BizTalk Server 
intemal infrastructure. You should not be using 
this in your code, as it is not supported. 
Provides an instance of a service, with a start 
and stop methods. 

Represents a suspended event for a service 
instance. 

Supports the BizTalk Server intemal 
infrastructure. You should not be using ts in 
your code, as it is not supported. 

Represents a message instance. 


Represents a message instance. 
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Exploring WMI Classes Using WMI Explorer 


"This open source tool is great tool ta explore all of the WMI classes registered in your 
‘computer. You can download it from https: //github. con/vinaypamnani /wnie2/ 
releases, 


How to Use WMI Explorer 


Executing WMI queries requires administrator privileges. Therefore, you must run the 
application as an administrator. 





1. Type the server name or leave it as the default if you are 


connecting to a local server. Click the Connect button, as shown in 
Figure 49, 
WMI Explorer 2002 (Administrator) 


Bie Launch Help 
Computer 


[eres] 
Figure 49. Connecting toa BizTalk Server environment 


your user has the proper rights, the tool will list all available namespaces, as shown. 
in Figure 4-10, 
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Figure 4-10. Selecting the BizTalk Server WMI namespace 


2. Double-click the ROOT\MicrosoftBizTalkServer namespace, not. 
ROOT\MicrosoftBizTalkServer\ns_409. The tool will show all 
avallable classes forthe BizTalk Server namespace, as shown in 
Figure 4-11. 
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Figure 4-11. Listing the available WMI classes for BizTalk Server 
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3. Double-click the MSBTS HastSetting lass. The tool will populate 
the instances, 


A, Notice that are several tabs related to the class: 


3. Instances Listsall instances related to the class, as shown in. 
Figure 4-12 
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Figure 4-12. Listing the available instances 


b. Properties Lists all available properties for those instances, 
as shown in Figure 4-13, 
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Figure 4-13. Listing the available properties 


c. Methods—The class SETS HostSettingdoesnot 
‘expose any methods. But if you explore the class MSBTS_ 
ServiceInstanceSuspendedtvent, you will see that the tool 
list all available methods, as shown in Figure 4-14. 


eed 


eos: 
— — 











—— — 
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Figure 4-14. Listing the available methods 
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you explore the MSTS HostSetting class using WMI Explorer, you can find that the 
following members are associated to each BizTalk Server host, as shown in Figure 4-15. 



































Figure 4-18. WMI Explorer host setting members 


These members directly represent all of the host settings exposed in the BizTalk 
Administration Console, 
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you explore the MSBT5_Hostsetting, you might notice that this class does not have 
any methods. The 
accessing the class directly. However 
default actions to all WMI classes. These operations are called Put Type options, and the 
definition is as follows: 





not create, delete, or update so you cannot perform any actions 








luckily for us, WMI has standard operations to 


[-PutType (None | Updateonly | Createonly )] 
Where: 
+ UpdateOnly updates an existing WMI instance. 


+ CreateOnly creates a new WML instance, 


Using PowerShell to Optimize the Environment 


Sandro Pereira, an integration MVP, published a great script to provide the initial BizTalk 
host separation policy, The script uses the MWI classes presented in this book and 
PowerShell. 
You can download it from here: 
inttps://gallery. technet .microsoft.com/PowerShell-to-Configure-ocees3e8 
In Sandro Pereitas words, the following hosts will be created: 
+ BizTalkServerTrackingost—A BizTalk Host that hosts tracking 
and is responsible for moving the DTA and BAM tracking data from 
the Message Box database to the BizTalk Tracking (DTA) and BAM 
Primary Import databases. This movement of racking data has an. 
impact on the performance of other BizTalk artifacts that are running. 
in the same host that is hosting tracks, Thus, you should use a 
dedicated host that does nothing but host tacking. 








Only the Allow Host Tracking option must be selected because 
only will use this host for tracking 

+ BizTalkServerReceiveHost—All options (Allow Host Tracking, 32-bits 
only, or Make This Default Host in the Group) should be unselected, 
"This host will be responsible for processing messages after they are 
picked up in a receive location. When a host contains a receiving item, 
such as a receive location (with a pipeline), the message decoding and 
decrypting occursin a pipeline within this host 
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All receive handlers, except the isolated ones like SOAP, HTTP, 
WCF-BasicHttp, WCE-WsHttp, or WCF-Customlsolated and the 
32-bit adapters (FTP, SQL, and POPS) will be configured for this 
host. This will mean also that all receive locations will run in this 
host instance. 


BizTalkServerReceive32Host—Has the same goal as the previous 


one; however, this must have the 32-bits only option selected so that 
we can run the 23-bits adapters. 











‘The receive handlers for the FTP, SQL, and POPS adapters will be 
configured for this host. 





BizTalkServerSendHost—All options (Allow Host Tracking, 32 
only or Make This Default Host in the Group) should be unselected, 
‘This host will be responsible for processing messages before they 
are sent out tothe send port, When a host contains sending item, 
such asa send port, the message signing and encryption occurs ina 
pipeline within this host. 


All send handlers, except 32-bit adapters like native SQL and FTP 
adapter, will be configured for this host. This will mean also that 
all send ports will run in this host instance. 





BizTalkServerSend32Host—Has the same goal as the previous one; 
however, this must have the 32-bits only option selected so that we 
can run the 32-bit adapters. 


‘The Send handlers for the FTP and SQL adapters will be 
configured for this host. 


BizTalkServerApplication—Only the 32-bits only option should 
be select in this host. This host will be responsible for processing 
messages based on the instructions in orchestrations that need to run. 
in 32-bit, 


BizTalkserverApplication&aHost—Only the Make this Default. 
Host in the Group option should be select in this host. This host will 
be responsible for processing messages based on the instructions in 
all or the most common orchestrations. 
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Using the BizTalk Host Configuration Settings File 


The XML BizTalkLowLatencyHostSettings. xml fle, located in the Chapter 4 source 
code file, contains the host configuration and associated host instance that you can reuse 
in your environment to import this configuration to any of your hosts. 





hal. version="1.0" encoding="utf-8"?> 
<Settings xnlns:xsis "http://w. u3 org/ 2001 /XHLSchena- instance’ 
mns sxsd»" http://'ne.u3.org/2001 XML Schema" 
dExportedaroupy BIZTALK2016: BIZTALKHGNTDB«/Exportedaroup» 
<GroupSettings> 
Setting Nane-"ALlowTrackingSettingsImport"»Truec/Setting» 
Setting Nanes"ConfigurationCacheRefreshInterval'»60c/Setting 
Setting Nane-"GlobalTrackingoption"»ic/Settingo 
Setting Nane="LuSFragnentSize">102400</Setting> 
«Setting Names" LUSThreshold"»1000000c/Setting* 
Setting Nane="PerfCounterCacheRefreshinterval">60</Setting> 
</GxoupSettings> 
diostSettings» 
«Host Nane-"BizTalkLodLatencyHost"» 
«Setting Nane-"ALlodftültipleResponses"»Truec/Settinge 
«Setting Nanes"AuthTrusted'»Falsec/Setting. 
«Setting Nane-"DBQueueSizeThreshold"»100000«/Setting» 
«Setting Nane-"DBSessionThreshold"»oc/Settingo. 
«Setting Nane="DehydrationBchavior">1</Setting> 
«Setting Nane»"DeliveryQueueSize">1000</Sett:ing> 
«Setting Nane="GlobalNenaryThreshold">0</Setting> 
«Setting Nane="HostTracking"»Falsec/Setting> 
«Setting Nane»"InflightMessageThreshold”»5000</Setting> 
«Setting Nane="TsHost32BitOnly">False</Setting> 
«Setting Nane=" Legacykhitespace"»Falsec/Setting> 
«Setting Nanes"NessageDel:iveryMaxinunDelay">300000</Setting> 
«Setting Nane»"NessageDeliiveryOverdriveFactor”>125¢/Setting> 
«Setting Nane-"HessageDeliverySanpleSpaceSize"»100c/Setting» 
«Setting Nane»"NessageDel:iverySampleSpaceWindow"»15000</Setting> 
«Setting Nanes"HessagePubl ishYaximunDelay" »300000«/Setting 
«Setting Nane="NessagePubl ishOverdriveFactor">125</Setting> 
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«Setting Nane="NessagePublishSanpleSpaceSize">100</Setting> 
«Setting Nane-"HessagePublishsanpleSpacekindon"»15000«/Setting» 
«Setting Nanes"NessagingMaxReceivernterval">50</Setting> 

«Setting Nane="NessagingReqRespTTL">3</Setting> 

«Setting Nane="NsgagentPerfCounterServiceClassID">0</Setting> 
«Setting Nane="ProcessMenoryThreshold"»100«/Setting> 

«Setting Nane="SubscriptionPauseAt">0</Setting> 

«Setting Nane="SubscriptionResunekt”»0</Setting> 

«Setting Nane="ThreadPoolSize”>40</Setting> 

«Setting Names "ThreadThreshold"»o«/Setting» 

«Setting Name-"ThrottlingBatchencryThresholdPercent"»ic/Setting. 
«Setting Nane="Thrott]ingDeliveryOverride"»2</Setting> 

«Setting Nane="ThrottLingDeliveryOverrideSeverity"»200</Setting> 
«Setting Nane="Thrott]ingLimitToTriggerGc"»80</Setting> 

«Setting Nane-"ThrottlingPublishOverride"»2«/Setting* 

«Setting Nane="Thrott]ingPubli shOverrideSeverity">100</Setting> 
«Setting Nane="ThrottLingSeverityDatabaseSize"»1</Setting> 
«Setting Nane="Thrott]ingSeverityIn¢LightMessage">75</Setting> 
Setting Nane="ThrottLingSeverityProcessHencry">500</Setting> 
«Setting Nane="ThrottLingSpoolMultipliex">10</Setting> 

«Setting Nane="Thrott]ingTrackingDataMultiplier"»10</Setting> 
«Setting Nanes"TineBasedaxThreshold'»1800c/Setting» 

«Setting Nane="TineBasedltinthreshold">1</Setting> 

«Setting Nane="UseDefaultAppDonainForIsolatedAdapter">Falsec/Setting> 
«Setting Nane»"XlanglaxReceiveInterval">50</Setting> 





«osto 


c Hostsettingso 
<HostInstancesettings> 
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cost Nane="BizTalkLowLatencyHost"> 





«Server Nane="Bi2Talkz016" 
<Setting Names"CLRMaxIOThreads">250</Setting> 
«Setting Names" CLRHaxiorkerThreads" 250c/Setting» 
«Setting Nane="CLRMinIOThreads">25</Setting> 
«Setting Names" CLRMinklorkerThreads"s25</Setting> 
<Setting Nanes"PhysicalMenoryHaxinalUsage">85</Setting> 
«Setting lanes" PhysicalHenoryOptinalUsage"»70c/Setting» 
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<Setting Names "VirtualMenoryMaximalUsage">85</Setting> 
<Setting Names "VirtualMenoryOptimalUsage">65</Setting> 
</Server> 
<Most> 
</MostInstanceSettings> 
<isettings> 





How to Import a Host Configuration File 


Follow these steps t 


ig the Administration Console 


importa host setting configuration file: 





1. Using the BizTalk Administration Console, access the Settings 
window of any hosts or by using the right pane of the console and 
clicking on the Settings option. 


2. Click on the Import button, as shown in Figure 4-16, 











* mes cea mm 
Figure 4-16. Importing host settings from a file 
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3, On the Import Settings Wizard, browse to the template file 
BizTalkLowLatencyHost. xnl and click Next, as shown in 
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Figure 4-17. Selecting the file settings location 
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4, The Import Wizard will now show a screen like the one in 


Figure 4-18. 
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Figure 4-18. Host mapping window 


5. The Destination-Host column is showing all the hosts of the. 
current environment, The idea here is that you are going to 
select the host configuration for the law latency host contained 
in the configurations file. The BizTalkLouLatencyHast does 
not necessary exist in your environment; you are ust using lt 
configuration as a template for any other host that needs to 
behave as low latency. 





For instance, imagine that in your case you want to 
apply all the low latency customizations to the host 
BizTalkServerApplicationost. To do that, select the 
BizTalkserverhpplicationHost, then click the Add button and 
select the BizTalkLowLatencyHost configuration host from the 
Select Source Entity window and click OK. 





205 


GHAPTERA  OPTMIZNG THE BIZTALK PLATFORM 
6. Now the Host Mapping window will look something similar to 
Figure 4-18. 
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Figure 4-19. Linking the destination host to the source host. 
Here you can see that you are going to assign the low latency 
settings to the BizTalkServerApplication host only. Click Next. 


7. The configuration file also contains low latency settings for host 
instances, therefore, i's time to do the same for the host instances 


of the host BizTalkServerApplication 
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On the Host Instance Mapping window, click on Add button, 
and select the BizTalkLowLatencyHost configuration host from 
the Select Source Entity window. Click OK. The Host Instance 
Mapping window should look something like Figure 4-20. 
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Figure 4-20. Mapping host instances 


D 


Click Next. 


Now the wizard shows the Import Summary window where you 
can see all the changes, as shown in Figure 4-21, 
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Figure 4-21. Import summary 


10. Click the Import button and wait until the wizard completes the 
import task. Once the process finishes, the Import Results window 
will be shown. Ifthere are any errors, you will see a red cross 
instead ofa green check. See Figure 4-22, 
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Figure 4-22, Import results 


1L. Now goto the BizTalk Administration Console, Click F5 to 
refresh the data and access the host settings window. Locate the 
BizTalkServerApplication host and check that the changes are 
applied. 


Message Box Database 


‘As you learned in Chapter 1, the Message Bos is used by the engine to store information 
such as messages, host queues, instances, subscription information, promoted 
properties, and temporary tracking data. Because of this, it becomes vital when it comes 
to performance. Depending on the hardware resources an SQL Server, single Message 
Box database environments can impose limitations on the number of transactions that 
your BizTalk group can process. 
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Especially for solutions where application priority levels are defined with the 
highest ranks for transactions levels, you should consider adding more Message Boxes 
proactively. That does not mean that you are going to add Message Boxes whenever 
a demanding application comes up. You need to keep in mind that there might be 
the possiblity that after adding that application, bottlenecks caused by Message Box 
processing can occur. Lucklly BizTalk Server offers the option to add more Message Box 
databases so all of them can work together to balance the load across of ll databases, 
Message Box databases have two roles: 
+ Master The master Message Box database evaluates and routes 
subscriptions to all Message Box databases in the BizTalk Server group. 


+ Publisher—The logic for publishing messages learned in Chapter 1. 


Alter you configure BizTalk Server, as there is only one Message Box database, it 
js automatically marked as master and publisher, When the Message Box becomes 

a bottleneck, itis usually because the publication role is causing that performance 

issue. The idea of adding more Message Boxes will be to distribute the publication role. 
across several databases. you add only one, it will not have an impressive impact 

‘on performance because the second Message Box will be the only one doing message 
publication. Therefore, to scale the Message Box layer, you will have to add a minimum 
‘of to Message Boxes. You will be disabling message publication to the original one, and 
now the BizTalk Server engine will distribute the load across the new databases 


Situations to Consider Additional Message Boxes 


An essential consideration when planning a BizTalk Server environment should be 
to determine the maximum sustainable throughput (MST) of the system. The MST of 
‘a BizTalk Server system is calculated as the highest load of messages that the BizTalk 
environment can process. When load exceeds MST, messages are queued in the Message 
Bos and transaction latency can increase. 

Ifthe MST is continuously exceeded over time, most likely you will see that 
the Spool, Host Queue, and Tracking data tables within the Message Box will show 
increasing trends during the time MST is exceeded. That can eventually raise a throttling 
‘condition based on database size and the BizTalk Server engine will put pressure to 
the message publication and the dequeue process. (Review Chapter 2 to extend the 
information about BizTalk Server throttling.) It is crucial here to compare the previous 
indicators along with hardware resource utilization, especially on SQL Server. 
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M BizTalk Server enters into the throttling state based on database size and CPU and. 
disk utilization in SQL Server are under normal values, this does not mean that MST 
has been exceeded, asit could be due to an increase in the load that does not cause a 
bottleneck yet In this case, you should consider tuning the threshold for message 
‘count in the database to allow SQL Server to process more messages. As discussed in 
‘Chapter 2, you have three options to update this setting: 
+ Message count in DB—BizTalk Server will enter the throttling state 
when any ofthe host message queues reach this threshold, By 
default, itis set to 50,000. 
+ Spool multiplierIfthe number of messages in the Spool table 
reaches the message count DB the Spool multiplier setting, BizTalk 
Server will enter the throttling state. 
+ Tracking multiplier IEthe number of messages in the tracking data. 
tables reaches the message count DB * the tracking multiplier setting, 
BizTalk Server will enter the throttling state. 





Keep in mind that by inereasing this threshold, the disk usage for Message Box 
database will also increase. Additionally, the engine will perform slower regarding. 
transaction latency as all the stored procedures that the engine is calling will take longer. 
to retrieve the data (high CPU usage on SQL Server could also be observed), 








Performance Indicators of an Exceeded MST 


‘The following performance counters will show increasing trends during the time the 
Message Box is causing a bottleneck when the MST has been exceeded (or is about to) 


^ BizTalk: Messaging Latency performance counters—If the Message 
Boxis performing slowly, usually you will also see that all latency 
performance counters show increasing trends during the same 
period. 





^ BizTalk: Message Bax: General Counters Spool Size and host queue 
counters-If the system is not able to keep up with the load, the Spool 
and Host Queue tables will show increasing trends as well. 
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. BizTalk: Message Box: General Counters Tracking data size—As 


tracking information itis first saved to the Message Box, you will 





see also an increasing trend ln this performance counters because. 
messages can be queued up in the Message Box when there is a 
processing bottleneck. 


+ Disk: Average Disk Queue Length When this counter repeatedly 
shows value of3 or more for the Message Box database, it can. 





indicate that there is a bottleneck on the Message Box, as disk 
contention is most likely to happen. 


‘Adding Two or More Message Box Databases 


After you analyze the data and you are sure that the Message Box database ls causing a 
bottleneck, you can follow these steps to add a new database 





Requirements 


+ The account used to perform the following steps must be part 
of the BizTalk Server Administrators groups and SQL Server 
Administrator role. 


^ As you should disable message publication in the primary Message 
Bax, no active subseriptions should be present at the moment of 
adding a message box. You will need to un-enlis all orchestrations 
and send ports before moving forward so this action requires. 
downtime. For simplification, you can entirely stop all BizTalk Server 
applications. Keep in mind that stopping host instances does notun- 
enlist subscribers 





Follow these steps: 
1. Open the BizTalk Administration Console. 
2. Under Platform Settings, click Message Boxes. 


3, Right-click Message Boxes and select New > Message Box, as 
shown in Figure 4-23, 
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Figure 4-23. Adding a new Message Box database option 


4. Inthe Message Box Properties dialog box, type the Server name 
ofthe SQL database in the SQL Server feld and then type the 
name of the new Message Box in the Database field. You can. 
use HessageBoxPublisher1 as these databases will be used for 
publishing role only. See Figure 4-24. Click OK. 
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Figure 4-24. Setting the properties for the new Message Box 


5. Repeat Steps 2-4 to add the a second Message Box database (you 
can use MessageBoxPublisher2 as the name for the second one). 


6. Now, go to the original Message Box and disable message 
publication. By doing this, you are distributing the message 
publication feature across the new databases. See Figure 4-25, 
Click OK. 
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Figure 4-25. Disable message publication for the original Message Box 


Now the original Message Box will be responsible for routing messages, and the 
other two will distribute the message publication. 


Optimizing Message Box Databases 


In previous sections, you learned that BizTalk Server uses Message Box database 
intensively and how you can add more Message Boxes when there is a processing 
bottleneck. As this database is the most important one regarding performance, you 
should proactively optimize it for BizTalk Server. 

When you configure BizTalk Server, the configuration wizard will set the Message 
Box database with the following settings: 





+ Max Degree of Parallelism—This configuration option contrals the 
number of processors that are used for the execution of a query in 
a parallel plan. The BizTalk Server engine uses lots af SQL Server. 
stored procedures that are developed in a way that if you change the 
default max degree of parallelism to a setting different than 1, several 
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locks and performance issues can arise (especially under high load). 
Therefore, do not change this value at all. This recommendation can 
be quite deceiving for most of SQL Server DBAs as the general best 
practice is to increase this seting based on the number of logical 
processors. 


+ Auto Update and Create Statistics Statisties for query optimization. 
data that contain statistical information about the distribution of 





values in one or more columns ofa table or indexed view. This 
information is used by the query optimizer to design a quality query 
plan. Again, the stored procedures used by the BizTalk Server engine 
are optimized for best results and changing these settings is not 
supported, 


The most common actions to improve Message Box databases are: 


+ Adding a minimum oftwo more message boxes, a topic covered in 
the previous section 


+ Separating the data fles and Log les. 
+ Creating distributed file groups. 
+ Using 4 KB NTFS allocation unit size 


‘Separating the Message Box Databases Data and Log Files 





Note The recommendation in this section can be also applied to tracking and 
business activity monitoring databases, 





‘The first time you configure BizTalk Server, using the Configuration Wizard, all the 
BizTalk Server databases are created with the default settings. Regarding the data and 
transaction log fles, the Configuration Wizard stores them in the same drive. Placing 
‘both data and log files on the same drive can cause contention for that delve, resulting 
in bad performance (especialy for BizTalk Server, as it is relaying fully on the Message 
Box database for processing messages). Placing the files on isolated drives allows the 
1/0 writes and reads to occur in parallel for both the data and the log files. Try to spread. 
the disk 1/0 across as many LUN as possible so that the storage hardware also performs. 
parallel processing. 
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Note To change the file locations, you need to bring the database offline. 





‘You can use this code to move the Message Box database files to a new location: 


USE master; 
a 
~- Return the logical file name. 
SELECT name, physical nane AS CurrentLocation, state desc 
FROM sys.naster files 
WHERE database id = DB ID(N'BizTalksghoxDb") 
AND type desc = N'LOG'; 


a 
ALTER DATABASE BirTalbMsgüoxDb SET OFFLINE; 
a 


Physically move the file to a new location. 
In the following statement, modify the path specified in FILENAME to 
the new location of the file on your server. 
ALTER DATABASE BizTalksgBoxDb 
MODIFY FILE ( NAME = BizTalkMsgBoxDb_Log, 

FILENAME = 'C:\NewLocation\BizTalkMsgBoxDb.1df*); 
MODIFY FILE ( NAME = BizTalkMsgBoxDb data, 

FILENAME = ‘Hs \NewLocation\BizTalkMsgBoxDb.ndf"); 








a 
ALTER DATABASE BirTalkMsgloxDb SET ONLINE; 
c 
Verify the new location. 
SELECT nane, physical nane AS CurrentLocation, state desc 
FROM sys.master files 
WHERE database id = DB ID(N'BizTallasgBoxDo") 
AND type desc = N'LOG'; 
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Creating File Groups 


Since the default BizTalk Server configuration creates the Message Box database using a 
single file in the default fle group, ifadding additional Message Boxes does not solve the 
disk contention problem, you should consider improving the file group distribution. You 
‘can add more SQL Server files and file groups to improve database performance because 
this functionality allows splitting database files across different disks, This will enable 
parallel read and write operations that eventually mean more transactions per second 
Tor BizTalk Server. Changing the default file group distribution has the benefit also of 
dedicating highs speed disks for the Spool table or the Host Queue tables of applications 
that have higher ranks of transaction priority levels (TL). 





Creating File Groups for the Message Box Database 


You can follow these steps to create a new file group distribution for the Message Box 
database: 


L Stop all BizTalk Server host instances, the Internet information. 
server, and the SQL Server agent. 


2, Open SQL Server Management Studio and connect to the instance 
holding the Message Box database. 


3, Using the Object Explorer, locate the Message Box database. 
Right-click it and select Properties, 


4. On the Database properties page, select the Filegroups option. 


5. Click on the Add Filegroup button. See Figure 4-26. 
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Figure 4-26, Adding a file group using the Database Properties window 


6, Repeat this operation for the number of files you want to create. 
7. Click the OK button. 


3. Using the Object Explores, locate the Message Box database. 
Right-click it and select Properties. 
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9. On the Database properties page, select the Files option and click 
on the Add button. See Figure 4-27, 
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Figure 4-27. Adding a file using the Database Properties window 


10. Serthe logical name to MessageBoxFilet and select the file group 
from the Filegroup dropdown box, as shown in Figure 4-28. 
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Figure 4-28. Adding a file and assign the file group 


1L. Select and appropriate initial size for the le You can use 520 (MB). 
12. Setthe Auto Growth setting to 100 MB. 


1a. Set the location toa different disk than the primary Message Box 
database file (whenever possible) 


14. Repeat Steps 9-13 to add the second file. 
15. cuckox 
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NTFS Allocation Unit Size 


As BizTalk Server is very transactional, it writes the data to SQL Server databases in a 
sequential way. For this reason, the best configuration is to set the NTFS allocation unit 
setting to 64 KB, 

You can change the unit allocation size when setting up a new disk partition using 
the disk management tool. In the New Volume Wizard, select 64 KB and click on the Next 
button and then on Finish. See Figure 4-29, 
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Figure 4-29. Setting the allocation unit size to 64 KB 


Trace Flag 1118 


your BizTalk Server environment is using SQL Server 2016, you do not need to enable 
the trace flag 1118, as the new default behavior of SQL Server is to use uniform extent 
allocation for frst eight data pages. 

HSQL Server 2014 is used, then you should implement this trace ag as it helps to 
reduce data contention, especially for the Message Box and Tracking databases. 

‘To enable the trace lag, open the SQL Server Configuration Manager, select the SQL 
Server instance, and type the trace flag under the Startup Parameters tab, as shown in. 
Figure 4-30. 
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Figure 4-30. Setting the trace flag 1118 


SQL Server Process Affinity 


SQL uses all CPUs available from the operating system. It ereates schedulers on all the 
CPUs to make the best use of the resources for any given workload. When multitasking, 
the operating system or other apps on the SQL Server can switch process threads from. 
‘one processor to another. SQL is a resource intensive app and so performance can be 
impacted when this occurs. To minimize, we can configure the processors in a way that 
all the SQL load will be directed to a pre-selected group of processors 
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For instance, ifyour environment has four cores, use this: 


ALTER SERVER CONFIGURATION 
SET PROCESS AFFINITY CPU = a to 2 


Note: CPU = 0, is the first CPU. 


By doing so, SQL Server will be using only the first three processors, leaving one for 
the rest of the applications and operating system. This will ensure that SQL Server will 
not be affected by threads being switched among the rest of processors. 

You can set this setting using the SQL Server Management Studio as well. Go to the 
properties ofthe SQL Server instance, and under the processor's category, uncheck the 
Automatically Set Processor Affinity Mask for All Processors checkbox. Then, select the 
processors you want to use in your environment and click the OK button, as shown in 
Figure 4-31. 

















Figure 4-31. Setting the processor affinity 
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Fixing Database Inconsistencies 
Database inconsistencies can occur in BizTalk Server forthe following reasons 

. Unknown product bugs 

«o Known issues 

+ Convoys leaving zombie messages 


When these problems arise, performance can be affected, as internal tables within 
Ihe Message Box and Tracking databases can grow because the SQL Server management 
jobs cannot clean messages that are inconsistent. 

"The Monitor BizTalk Server (BizTalklgntDb) job alerts you when these 
inconsistencies occur. This job runs weekly on Sundays and it will generate an error 
‘when database inconsistencies are found, 

Figure 4-32 shows an example of the output when the job finds database 
inconsistencies, 








Figure 4-32, Viewing the monitor jab history 
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For the latest BizTalk Server versions, you can fx these inconsistencies using the 
BizTalk health monitor tool. 


Follow these steps: 
1. Open the BizTalk Health Monitor toal, 


2 Right-click at the default profile level (if you do not have a custom. 
profile) and select Analyze Now, as shown in Figure 4-33. 


(i BizTalkHealthMonitor - [Console Root\BizTalk Health Monitor\Defau 
File Action View Help 
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IE} BizTalk Server Administration 
8, BizTalk Health Monitor 
> lil Monitoring profiles 
sg Default profile [BIZTAN 
> Ta Performance 
aman O° T roepane > 
> ll Event Viewer (Loca) Profile settings 





@ Dashboard 










Duplicate profile > 
Delete 

Refresh 

Help 


jp ‘erated int 





Figure 4-33. Running the default profile 
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3. "The tool will check the BizTalk Server environment. 


4. Wait until the process finishes. 
5, In the dashboard report, click on the Message Box database group 
and locate the MsgBox database integrity section, as shown in 

Figure 4-34, 
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Figure 4-34. Looking for database inconsistencies Message Box 


6, the tool finds any inconsistencies, they will be listed here. 


7. Clickon the Maintenance section on the left pane and select the 
default profile on the dropdown box, as shown in Figure 4-35, 
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Figure 4-35. Connecting to the BizTalk Server Management database to fix 
database inconsistencies 


Fixing database inconsistencies requires the following actions from 
your side: 

+ Generate a backup ofthe BizTalk Server databases, 

^o Stop all BizTalk Server host instances. 

+ Stop the SQL Server agent. 


3. Once you check all the confirmation checkboxes, clickon 
Connect button and wait until the connection to BizTalk Server is 
established. 


3. From the task type dropdown box, select the From the Latest BHM 
Report option, as shown in Figure 4-36. 
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Figure 4-36, Selecting inconsistencies from the latest BHM Report 


10, The tool will look for all of the Inconsistencies found in the 
previous report and it will populate the Task List dropdown box. 


1L. Selecta task from the Task List dropdown box and click the 
Execute Task button, as shown in Figure 4-37. 
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Figure 4-37. Executing a task list using BHM tool 


12. Repeat the Step 11 until you have fixed all the inconsistencies 
provided by the BHM tool. 
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Scaling the BizTalk Server Group 


You can scale the BizTalk system by adding hardware resources to the existing BizTalk 
Server or adding more servers to the group. Adding more servers to the BizTalk group 
should be done only in the cases where the Message Box database server is performing 
efficiently, because you are going to create more host instances that will increase the 
load on the Message Box database. 

For instance, imagine that your environment has two BizTalk Server nodes and you. 
decide to add two more to increase the throughput of the system. You perform all the 
testing and you find out that the throughput has increased by 30%. Your production 
‘environment is now able to process more messages. After a while, you have a new. 
requirement and you decide to add two more BizTalk Servers. However, this time you 
observe that throughput has indeed decreased by 1555. What could be causing this issue? 








^^ Message Box has become a bottleneck. 
+ SQLServer resources consumption is over acceptable thresholds. 
^O The network is saturated with the new load, 


The previous example is based on real customer scenarios. The takeaway here is 
that whenever you want to increase the processing power of the BizTalk Server layer by 
scaling or tuning host performance settings, you should proactively monitor your SQL 
Server environment and pay attention to any of the performance indicators that will 

tell you whether or not SQL Server is performing efficiently with the newarchitecture/ 
‘configuration. Review the Message Box section to learn how to detect a bottleneck on the 
Message Box database, 





Microsoft Distributed Transaction Coordinator 


MSDTC is the Microsoft Distributed Transaction Coordinator. MSDTC provides the 
functionality to ensure complete transactions across a distributed environment, which 
means across two or more networked computers. 

Je may help to think in a typical database transaction to imagine what is happer 
under the hood: 





BEGIN TRANSACTION 
DO SOMETHING across several DIFFERENT MACHINES (MAY USE LOCAL TRANSACTIONS) 


CONIT TRANSACTION (OR ROLLBACK) 
zm 
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In this way, database and distributed transactions enforce the ACID properties— 
Atomicity, Consistency, Isolation, and Durability. 

By default, network DTC access is disabled, Without network DTC access on the 
server, applications can only use transactions that stay on the local computer. For 
instance, transactions cannot flow from a local computer toa database that runs on a 
separate computer. Since BizTalk leverages several different databases, and in most of 
the cases spans multiple servers, MSDTC is used extensively to communicate with SQL 
Server, especially if using adapters supporting transactions, such as classic SQL Server, 
WCE SQL Server, and MQSeries. 


How DTC Works 


‘The basic conceptis that Server A initiates work in a transactional context (Initiates a 
transaction) To complete the operation, it must do something in a SQL Server database 
running on Server B. I then connects to Server B hosting the SQL Server database 

and does some work. Imagine the operation on the SQL Server machine fails due to 

a problem so that the transaction initiated on the SQL Server machine must be rolled 
back as well as the operations on Server A. 

Each computer ina distributed transaction, has its resources and participates as 
an element in the global transaction that must be committed or aborted across all the 
servers involved, MSDTC performs the coordination role for the components (and 
machines) and decides ifa global transaction is successfully committed or must be 
tolled back, 

In general, DIC uses protocol based on two phases: 








1. Applications call the transaction manager (DTC) to begin a 
transaction. At this point, the transaction is no longer local, and 
DTC coordinates the state. As BizTalk Server Inserts fragments 
in the context ofa new DTC transaction, itis imperative that you 
understand this concept because ifyou increase the number of 
fragments, SQL Server will use more resources to allocate the 
transaction, 


2. When the application has prepared its changes, it asks the 
transaction manager to commit the transaction, The transaction 
manager keeps a sequential transaction log, so its commit or 
abort decisions are durable. The transaction log isa physical log 
file on disk 
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"This log fle is more important than it seems since it can become a disk bottleneck 
in BizTalk Server systems. If your platform is receiving 100 messages per second 24 
hours per day, that means 8,640,000 messages per day. Now, imagine that all messages 
are the same size and that every message is split into 10 fragments. If the disk used to 
store the DTC log is not dedicated and, for instance, shared with the Windows system 
drive, the disk could potentially experiment high witing-latency, affecting overall disk 
performance. In this scenario, the Windows operating system could perform slow IO. 





operations, 
scenario). 


id that would affect performance globally (especially under a high load 





Note The DTC log file should be dedicated in an isolated disk to prevent 
serious performance issues. If DTC fails to enlist a new transaction because of 
performance issues, it will timeout, and BizTalk Messages will not be published 
(if fragmentation is in place). 





Number of Simultaneous Connections of HTTP Adapters 


By default all the HTTP based adapters will establish only two simultaneous 
connections from each server with BizTalk Server installed, This specification provides 
clear limitations, especially for low latency or high throughput applications. 








Note This setting conforms to the IETF RFC for the HTTP 1.1 specification. 
mot a BizTalk Server limitation, and although it is suitable for user scenarios, it is 
mot optimized for high load scenarios. 





"To change the default behavior, you can add the connectionanagenent section to 
the BizTalk: 





«ver configuration file: 


«configuration» 
<systen.net> 
«connectionlanagenent» 
«add addvess="htty 
maxconnection-"i; 


wi. YourDestiationServicelRL.con" 
^ 
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<add adéress-"http://wwi, YourDestiationServiceURL2.con" 
maxconnection«"i2" /> 





<add address="*" maxconnection«"8" /» 





</connectionanagenent> 
</system.net> 
«configuration. 


You can add the URI of the consumed services that use the HTTP protocol. You can. 
access the adapter configuration at the send port level to retrieve the URI ofthe service, 
as shown in Figure 4-28, 
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Figure 4-38. Retrieving the Send port URI 
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<add address="*" maxconnection: 


Note that the setting: 


a" /> 





Uses an asterisk asthe destination URL. You will use this section so that BizTalk 
Server will use this number of concurrent connections per BizTalk Server to send 


messages for the rest of the non-speciied locations. 





Warning 


Increasing this setting can flood the destination system. This situation. 


is undesired because the consumed service might not have enough resources 
to respond on time and frequent timeouts can occur. In this case, the number of 


suspended messages can increase exponentially 





Windows Communication Foundation Throttling 


WCF service 





n implementa behavior that is called service throttling, This behavior. 


allows you to throttle WCF requests to save hardware resources and to avoid flooding 


destination systems, 
The throtling behavior has the following configurable thresholds: 


+ maxConcurrentCalls-This threshold is used to establish the 
maximum number of messages that the WFC service will process 
When that number is reached, the next 
new calls will be executed only when the throuling condition is 
alleviated, 








+ maxConcurrentSessions—! 





setting controls the number of 


calls are queued, and 


maximum concurrent sessions that WCF will allow before starting a 


throttling condition. 


+ maxConcurrentInstances—Similar to BizTalk Server, WFC 
sessions and messages has a contest associated with it, and it 
is called InstanceContext. You can use this threshold to limit 
the number of simultaneous InstanceContext objects that the 
WCF service will process, The default value should be the sum of 
snaxConcurrentSessions and maxConcurrentCalls. 
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"The following table shows the recommended values for BizTalk Server: 


Threshold Recommended Value 
maxConcurrentCalls dé'number of cores 
maxConcurrentessions —— lémumberofcores 
mxConcurrentInstances —— maxConcurrentCalls + 
maxConcurrentSessions. 


"To enable this behavior, you need to edit the WFC configuration file and add the 
serviceThrottling section to the ServiceBehaviors section. 


 dbehaviorso 
«servicelehaviors» 





behavior nane-"Throttled'» 
«servicehetadata httpGetEnabled="true"/> 
«serviceDebug includebxceptionDetailInFaults»"false'/» 
«serviceThrottling maxConcurrentCalls-"200 
maxConcurrentSessions«"200" maxConcurrentInstances ="400" /» 








behavior» 
«/serviceliehaviors» 
behaviors» 


Documenting Applications 


In this section you learn about documenting your BizTalk Server solutions, which isa 
topic that provides a framework not only to understand the solution itself but also to size 
the BizTalk Server databases accordingly 

‘The idea is that every time you start a new BizTalk Server project, you should have 
a documentation template that should be fulfilled and maintained throughout all the 
phases ofthe project. 

A proper BizTalk Sen 











+ solution documentation should include this information: 


‘+ General application information, such as the name, development 
creation time, documentation version, changes, ete. 


+ Application priority level definitions 
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+ Performance SLA definitions 
+ Business ows 
+ Deployment and troubleshooting instructions 


‘This book uses Excel as an example of how you can document this information. [have 
chosen Excel, because the document is more ofa calculator that will help you size a new 
‘BizTalk application properly. The idea is that every time you start a new project you will 
‘have to fli this document until you completely finish the development. It exposes a clear 
picture of the application performance requirements, so you can apply specifie settings 
and development techniques in order to achieve the performance SLA. Additionally it 
calculates the growth of the Message Box and Tracking databases when you enable tacking 
for the specific artifacts. Even though calculations are not exact, it will give you a good idea 
Whenever you have to allocate more disk resources to your production SQL Server databases. 


The Excel Sheet 

The Excel file is divided into the following sheets: 
+ Application form 
+ lows 
+ Sizing Message Hox and Tracking DTA 


+ Data source 


Application Form 


‘This sheet is designed to specify al the application configuration, as shown in Figure 4-39. 














Figure 4-39. Examining the application general configuration 
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Application Priority Levels Section 


As discussed in this chapter, application priority levels can be used for sizing and to 


adjust BizTalk Server performance settings. In this section af the application form, you 


should classify your application using these concepts, as shown in Figure 4-40, 
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Figure 4-40. Examining the application priority levels. 


Performance Data Section 


"This section is designed to document the relevant performance data that has an impact 
when sizing the BizTalk Server databases. 
The following information is included: 


Number of transactions —These definitions should be agreeable to 
the business decision makers during the initial phases of the project 
and itis directly related to the performance SLA definitions. 
+ Number of incoming business transactions This value 


represents the number of incoming business transactions unde 
normal load. 





+ Number of outgoing business transactions This value 
represents the number of outgoing business transactions under. 
‘normal load. 


+ Max number of live transactions This value represents the 
‘maximum number of incoming business transactions under 
high load. 





Number of hosts—This definition represents the number of hosts that 
this BizTalk Server application is using. Refer to the Host Architecture. 
section in this book. 
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+ Number of subscriptions —The number af active subscriptions 
hasan impact on the Message Box sizing. This tool calculates the 
maximum size possible per subscription, so you have to prov 
the number of subscriptions that the application is using. The book 
Includes a SQL Server script to get this information. Unzip the APRESS 
folder and navigate to Chapter 4\scripts\Nunber of active 
subscriptions. Then open theNunber of Active Subscriptions 
per application. sql file 





Change the application name to the one you want to get the 
number of subscriptions, detailed in bold in the following code: 


USE BizTalkfsghoxDb 


SELECT count(*) as [Number of active subscriptions] 
FROM Services s WITH(NOLOCK) 

LEFT OUTER JOIN Modules m WITH(NOLOCK) ON s.nModuleID = 
m.noduleTD 

LEFT OUTER JOIN Subscription sub WITH(NOLOCK) ON s.uidServiceID = 
sub.uidServiceID 

LEFT OUTER JOIN PredicateGroup pg WITH(NOLOCK) ON sub. 
uidPredicateGroupID = pg.uidPredicateORGroupID 

LEFT OUTER JOIN FirstPassPredicates p WITH(NOLOCK) ON 
pg.uidPredicateANDGroupID = fp.uidPredicateGroupID 

LEFT OUTER JOIN EqualsPredicates eq WITH(NOLOCK) ON 
pg.uidPredicateANDGroupID = eq.uidPredicateGroupI0 

LEFT OUTER JOIN EqualsPredicates2ndPass eq2 WITH(NOLOCK) ON 
pg.uidPredicateANDGroupID = eq2.uidPredicateGroupID 

LEFT OUTER JOIN BizTalkMgntDb. dbo.bt DocurentSpec ds WITH(NOLOCK) 
ON eq.uidPropID = ds.id 

LEFT OUTER JOIN BizTalikgntDb. dbo. bts Orchestration o WITH(NOLOCK) 
ON s.uidserviceld = o.uidGUTD 

where m.nvellane = 'Bookürdershpplicstio 





+ Maximum number of scheduled transactions—If the application has 
receive or sending ports with schedules enabled, then you will need 
to specify the maximum numberof scheduled message instances. 
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+ Convoy pattern implemented—Convoys have their own set af tables 
within the Message Box and depending on the size, they can havea 
significant impact on the database sizing. 


^ Maximum number of instances ofthe convoy—This is the maximum 
number of instances that the convoy will handle. 


Figure 4-41 shows an example ofthe performance data section for the BookOrders 
application. 














Figure 4-41. Examining the performance data 


Documented Flows Section 


In this section of the application form, you should name all the business flows that the 
application is dealing with. Later, you will have to add an Excel sheet per business flow so 
that everything gets document properly. See Figure 4-42, 








Figure 4-42. Examining the performance message flows 
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Flow Forms 


You should create detailed documentation of the lows that your BizTalk Server 
application is using. You have to insert a sheet per every flow documented in the 
Application Form sheet. 

Figure 4-43 shows the flow call to CRM, which is the only Bow implemented in the 
application. 























Figure 4-43. Examining a message flow definition 


Sizing Message Box and DTA 


Iis very important that you define all messages, with message types and orchestrations, 
because you will have to populate the Sizing Message Box and DTA sheet with this 
information. 
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Configuration Item Section 


You can find the configuration item section on the top-left corner of the Excel sheet, as 
shown in Figure 4-44, 


Purgngeyde (days) 7 
‘sizeof promoted properties (KB) noo 
‘Size of orchestration shapes (KB) 0.076 
vents size (KB) 0224609375 


Figure4-44. Examining the configuration item section 


H contains these settings: 


+ Purging cycle days—This is the number hard days that the purging 
job is used to keep data. Refer to the Purging Tracking Job section in. 
the book. 


+ Size of promoted properties— This is the average size in KB of the 
promoted properties. Do not change this value unless you have 
large promoted properties. 





ery 


^ Size of orchestration shapes—This setting should not be changed 
as itis the internal average size of orchestration shapes within the 
Message Box. 





+ Events size This set internal 





ng should not be changed as it is the 
average size of pipeline and orchestration events within the 
Message Box. 





Calculated Sizing Data Section 


Th 


culated sizing data section shows the data sizes that your application will use and 





leves this information by calculating all the options specified in the elements table. 
Figure 4-45 shows an example of calculated data for the BookOrders application. 
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Extra size tracking dat T 
MessageBox (MB) 26025 
—— 43.50 
Max Load forall hosts ra 
Max Load for subscriptions. ss 
Max Number of suspended 30250 
Max Load for spool 0.68 
Max Load for Instances 190 
Max Load for Scheduled om 
Max load Binary message data. DI 
Max toad Convoy 520 


jebox (MB) 





Figure 4-5. Examining the calculated 





Extra Size Tracking data: 


^^ MessageBox (MB) This is the number of megabytes required to 
hold all the tracking data within the Message Box. Remember that 
the tracking feature inserts first tracking data to the Message Box 
database and then moves it to the Tracking database 

+ Tracking DTADB (GB) This is the numberof gigabytes that your 
application will generate in the Tracking database (based on the 
purging cycle specified in the purging job). 


Extra Size for messaging (MB) 
These fields calculate the maximum size ofthe following elements when the application 
is running under the maximum load possible. The maximum load possible Is obtained 
from the application form sheet called Max Number of Live Transactions: 
+ Maxload for all hosts—This is the maximum size of the host queue 
tables 
+ Maxload for subscriptions —Thisis the maximum size ofthe 


subscription tables. 
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Max number of suspended—This is the maximum size of suspended 

messages 

Max load for Spool Ihe maximum size of the Spool table. 

Max load for instances The maximum size of the instances table. 


Max load for scheduled —The maximum size of the scheduled 
Instances. 


Max load for binary message data—The maximum binary size ofthe 
messages tables. 


Max load convoy—The maximum size ofa convoy. 


Element Definitions 


In this table you define messages, orchestrations, and pipelines per flow. 


Steps for Adding Messages 


Note that here you have to add messages or send through ports and messages that are 
processed within the orchestrations. 


1 


2 


‘Type the message name in the Element Name field. 





ninute. 





Specify the transactions pe 


Specify the transactions per purging cycle. By default, this field 
is calculated by multiplying the number of transactions pe 
minute by the number of purging cycle days (considering that the 
application will work only eight hours per day). You can overwrite 
this value with the number you want. For instance, imagine in 
a particular flow, BizTalk can process 100 instances per minute. 
Seven days of tracked data, for eight hours of messages processed 
per day means 336,000 tracked messages for those seven days. 





Select Message as the type of artifact. 


Do not fill in the #Receive and #send shapes columns, as those are 
for orchestrations. 


Fill in the size of th 





message in KB, 
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10. 


12, 


1 


n. 


Set the maximum number of suspended messages related to this 
message. 


Set the number of promoted properties 
Do not fill in the number of shapes, as those are for orchestrations. 


the message is received or sent through a receive port, specify i 
the port has the following tracking properties enabled: 


^o Message Body Before 
^o Message Body After 
^o Properties Before 
^o Properties After 


Ifthe message ls used in an orchestration, specify these values 
when the orchestration is tracking message bodies and properties: 


«o Message Body Before > Orchestration tracking settings for track 
message bodies before orchestration processing. 


^o Message Body After > Orchestration tracking settings for track. 
message bodies after orchestration processing. 


+ Properties Before » Orchestration tracking settings for track 
message properties Incoming messages. 


+ Properties After > Orchestration tracking settings for track 
message properties outgoing messages. 


Do not fll in the Events Tracking section, as those are for pipelines 
and orchestrations. 


Observe how the calculated data section changes along with the 
introduced data, 


Set the name ofthe flow this message is involved with. 
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Steps tor Adding Orchestrations 


Follow these steps to add an orchestration to the documentation: 


10. 





Type the orchestration name in the Element Name field. 


Specify the transactions per minute. 





Specify the transactions per purging cycle. By default, this 
Beld is calculated multiplying the number of transactions per 
minute per the number of purging cycle days (considering that 
the application will work only eight hours per day). You can. 
overwrite this value with the number you wan. For Instanes 
imagine in a particular flow, BizTalk can start 100 orchestration, 
instances per minute, Seven days of tracked data, for eight 
hours of orchestration processed per day means 33,600 tracked: 
orchestrations for those seven days. 











Select Orchestration as the type of artifact. 


Fill in the #Receive and send shapes, These are 





he number of 
receive and send shapes executed by the orchestration. 


Set the maximum number of suspended orchestrations that the 
application can generate. 


Do not fill in the number of promoted properties, as this is for 
messages 


Within the orchestration, count the number of shapes and fil in 
the number of shapes with that value. 


Do not fll in the Data Tracking section, as those settings are for. 
messages, 


Set Events Tracking section when the orchestration has tracking 
enabled for these options: 


+ Orchestration start/end 
^o Messages send and receive 


+ Shape start/end 


E 
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Follow thes 


1L. Observe how the calculated data section changes along with the 
introduced data, 


12, Setthe name of the flow this orchestration is involved with. 





steps to set up a pipeline: 
1. Type the pipeline name in the Element Name field. 
2, Specify the transactions per minute. 


3. Specify the transactions per purging eyele. By default, this eld 
is calculated multiplying the number of transactions per minute 
per the number of purging cycle days (considering that the 
application will work only eight hours per day). You can overwrite 
this value with the number you want, For instance, imagine in 
a particular flow, BizTalk can start 100 pipeline instances per 
minute. Seven days of tracked data, fr eight hours of pipeline 
processed per day means 360,000 tracked pipelines for those 
seven days. 


4. Select Pipeline as the ype of artifact. 


5. Donotsetthe esend and &tecelve shapes, as this is for. 
orchestrations. 


6. Do not fill in the size of the message in KB, as this property is only 
for messages 


7. Ignore the maximum number of suspended, 


8, Do not set the number of promoted properties, as those are 
related to messages. 


9. Do not fil in rhe number of shapes, as those are for orchestrations. 


10, Set Events Tracking section when the pipeline has tracking. 
enabled for these options 


+ Port start and event > Start/end 
+ Messages send and receive events > Messages send and receive 


‘+ Shape start/end. Not applicable to pipelines, only to 
orchestrations 
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11. Observe how the calculated data section changes along with the 
introduced data, 


12. Setthe name of the flow that this pipeline is involved with. 


After you document all the lows, with all messages, orchestrations, and pipe! 





Excel will calculate the total extra size required in the Message Box and Tracking. 
databases so that you can size the adequate size for both databases. See Figure 4-46. 





Messagefcx (MB) 26025 
—* OTADS (6 43.50 
Max Load forall hosts 351 
Max Load for subscriptions s9 
Max Number of suspended. 30730 
Max Load for Spool 065 
Max Load for instances 190 
Max Load for Scheduled oo 
Max load Binary message data 98.83 
Max Load Convoy 9.20 





Figure 4-46, Examining the required extra size for the databases 


Summary 


In this chapter you learned how to categorize application based on application priority 
levels. This action will enable you to decide how the application will behave in terms of 
business requirements, numberof transactions, high availability, performance behavior, 
and the maturity of the application. 

Once you have adjusted the application priority levels to your own needs or based 
on your experience, you can use this concept to create a solid host separation policy, 
tracking, and performance settings 
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‘There are hundreds of BizTalk Server performance settings and, in this chapter, 
you go through the ones that you can certainly use proactively to reduce the chances of 
running into performance issues within your BizTalk and SQL Server platform. 

In the nest chapter, you learn how to use the most common techniques to 
Instrument your BizTalk Server solutions. This topie is essential to reducing 


troubleshooting time and therefore, decreasing application downtime, as you can find. 
bugs faster. 
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CHAPTER 5 


Instrumenting BizTalk 
Solutions 


In software development, instrumentation refers to the ability ofan application to 
monitor business processes, diagnose issues by revealing debugging information, and 
‘expose performance indicators. 


The BizTalk Server product extensively covers all these requireme 





sy 





+ Exposing performance counters. 


* Re 





'aling debugging information thought ETW Traces 
^o Writing events to the Event Log 

+ Creating raw BizTalk Engine traces 

^o DTA tracking (operational information) 

+ Custom Business Activity Monitoring (business tracking) 


To achi 





complex business processes requirements, the majority of BizTalk 





Server applications of today run custom code that, by default, s not implementing any 





instrumentation. As detailed in previous chapters, developers can create custom code in. 
any of the following BizTalk e 








^o Custom pipeline components 
. Orchestrations 

+ Custom components 

«o Maps 


^o Business rules. 


29 





CHAPTERS INSTRUMENTING BIZTALK SOLUTIONS 


Therefore, you as a developer should instrument your BizTalk applications by 
Inserting trace statements in relevant areas of your custom code. 

Efficiently implementing custom tracing mechanisms for BizTalk Server could be 
very complicated, as the engine might be distributed across several BizTalk Servers and, 
apart from particular cases like orchestrations with a correlated interchange or order 
deliver 





y convoys, you will never know in which server a particular service instance is 
y one BizTalk Server, all racing and debugging. 
technologies like writing debugging information to local drives, become overly 
‘complicated to analyze because th 
Additionally, depending on 
concurrent services per second, th 


running, Unless your envionment has only 








several factors like BizTalk load, number of issues and 
cost of implementing custom tracing mechanisms 
‘could be potentially remarkably high regarding CPU, memory, and IO consumption. 








Luckily for us, BizTalk Server is using already several tracing capabilities from Event 
‘Tracing for Windows (ETW) to track what is happening under the hood of most of the 
BizTalk Server Engine elements, like the End Point manager, the Message Agent, and 
most ofthe Transport Adapters. 


Instrumenting Using Event Tracing for Windows 


Eves 





Tracing for Windows (ETW) is a fabulous, efficient tracing system used by 
Windows Operating Systems. It exposes an API that is divided into he following 
components: 


+ Controllers can be custom applications that define the location of the 
‘output log file and can also start and stop event tracing sessions. 





+ Providers are the ones providing events to the tracing sessions, There 
are several types of event providers but, the most important ones for 
you, asa BizTalk Developer, are TraceLogging providers since BizTalk 


server implements tracing using this method. 





+ Consumers are the components that read data from memory or the 
defined log. 
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Figure 5-1 illustrates the relationship among these three components, 
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Figure 5-1. Diagram of the ETW architecture 


Does this sound familiar to you? If you read the Chapter 1, you can also see that the 
ETWis based on a publish/subscribe model, The same as BizTalk Server! 
BizTalk Server encapsulates the ETW functionality in two assemblies: 


+ Microsoft. BizTalk. 





agnostics. dll 
+ Microsoft. BizTalk. Tracing.dll 


Icuses the TraceProvider class to access most of the methods to interact with the 


ErWengine, 


E 
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Along time ago the Microsoft BizTalk CAT Team had tested this solution and compared 

to other mechanisms like Enterprise Library, logtne, orSysten. Diagnosis. Trace, 

The ETW showed meaningfully better performance. The study was blogged in the MSDN 

Blog post: "Best Practices for Instrumenting High-Performance BizTalk Solutions” 
https://blogs.nsdn.nicrosoft. con/asgisv/2010/05/11/best-practices-for- 

instrunenting-high-perfornance-biztalk-solutions/ 


The BizTalk CAT Framework for ETW 


Back in 2010, the Windows Server AppFabric Customer Advisory Team created a 
Solution framework that enables BizTalk Server custom code ta start new trace sessions 
in the ETW engine. You can access the solution projects from the companion ZIP file. 
‘Once extracted, you will se it under C: \Appres \Chapter5\CATFrameworksETH, 
Unfortunately, at the moment of writing this book, the solution projects have not 
been updated to run in 48x Net framework versions. Luckily, the updating process itis 
quite easy Ifyou perform the following steps: 
1, Extract the provided Zip file at C:\APRESS\Chapters\ 
CATFrameworkaETW\CATOriginal .zip. 
2, All files are read-only so you must remove the Read Only attribute 
10 all fles. Ifyou miss this step, Visual Studio will not beable to 
migrate the solution in a later step, 
3. Find the solution fle Best Practices for Instrunenting High- 
Performance BizTalk Solutions.s1n and open it in Notepad. 





A. Locate the lines shown in Figure 5- 
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Figure 5-2. Locating the TPS settings 





And delete ll o it, because if not, the Upgrade Wizard will fall (as the original 
solution was linked to a local TES server). 


5. Save the file. 
6. Now you can open the solution file with Visual Studio 2015. 


On the Review project and solution changes screen, check all projects 
and dick OK. See Figure 5-3. 
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Figure 5-3. Review project and solution changes screen 


7. "Ihe Visual Studio migration wizard will update the solution. 


Depending on your machine configuration, you could potentially have different. 


issues, as shown in Figure 5-4. 


Figure 5-4. Possible migration issues 
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Projects are pointing to Framework 2.0 and 3.5. Ifyou have these 
versions installed, you might not get any warnings. However, BizTalk 
Server 2016 works only with 4.6x versions so we will need to change 
the target for the three projects. 


Remove the Microsoft. BizTalk. CAT. BestPractices.Sanples. 
UnitTests project from the solution. 


For each project, change the Target Framework property to 4.6, as 
shown in Figure 5-5. 
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Figure s·s. Changing the target framework version 

10, Choose Save All and rebuild the solution. 

1. You can get the following error: 
Error (50535 'ConponentTraceProvider' does not 
inplenent interface member 'IConponentTraceProvider. 
TraceInfo(Funccstring>)' (new interface on framework 4.6) 

12. Double-click the error: Visual Studio will ake you to the 
ConponentTraceProvider definition. 


13, Right-click on ConponentTraceProvider and choose Quick 
Actions and Refactorings. See Figure 5-6, 


mm applicati grate gs 5 
essen Cole Map us 











Figure 5-6. Quick actions and refactorings 
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44. Select the Implement interface option, as shown in Figure 5-7. 


public sealed class ComponentTraceProvider : IGONBONUNETESEREIGUIdGF 


£50535 CormporertTaeorovidr doet not implement itr 
ST melemen imera opii — menbersComponentInesPrvider Tacana Fune eei 








LET 
s | 
$5 
a | 
E 
| — 





— — 





— in Document | Project Sluion 
Figure 5-7. Implementing the missing interface. 
15. Choose Save All and rebuild the solution. The rebuild action will 
succeed now. 


16. Athe project level, sign the assembly by creating a strong name 
key. Access the project properties, go to the signing section, and 
enable the checkbox Sign the Assembly. Create a new one and set. 
the key file name. Then click OK button, as shown in Figure 5-8, 
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Figure 5-8. Generating a strong file name for the project 


Enabling an Existing BizTalk Solution to Work with ETW 


Now that you have a compatible version for BizTalk Server 2016, you can use the 
instrumentation framework by referencing the assembly Microsoft. BizTalk.CAT, 
BestPractices. Franesork. dll to your project, As this framework assembly uses 
BizTalkTracing.dl, you must manually deploy it to your BizTalk Server environment 
also. The BizTalk Server tracing component is installed into the GAC, and you can find a 
reference to it here: 





C:\Windows\Microsoft,NET\assenbly\GAC_MSTLMicrosoft,BizTalk.Tracing 


In this case, we are going to use the BookOrders application and it includes a 





reference to the BizTalk tracking assembly already. For your future projects, remember to 
reference this assembly also; otherwise, the application will not work. 
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Follow these steps: 


L Once the solution is deployed, add the assembly as a resource, as 
shown in Figure 5-9. 











— — 


Figure 5-9, Adding the tracing component to the application 


Using the CAT Framework to Trace Orchestrations 


ible ETW in orchestrations, 





In this section, you will go through a step-by-step guide to e 
Follow these steps: 

1 Ifyou went through previous exercises, you will have a 
BookOrdersApplication. Remove it manually, otherwise you will 
get unexpected errors. 

2, This walkthrough uses the base book order solution contained in 
the companion source code folder for Chapter 5. Before moving. 
forward, ereate a back copy of the original file so you can revert 
the solution to the original state in case something goes wrong. 
Using Windows Explorer, navigate to C:\APRESS\Chapter5 and 
make a copy of the BookOrdersSolutionBase folder. 


3. Start Visual Studio with elevated privileges (run as Administrator, 
otherwise the deployment will fail) and open the C: MPRESSV 
Chapter5\BookOrdersSolut ionBase\BookOrdersSolution.sin 
solution, 


4, Open and explore the orchestration orcProcessBookOrders, 
located in the orchestrations folder. See Figure 5-10. 


CHAPTERS INSTRUMENTNG BIZTALK SOLUTIONS 











Ine 





Figure 5-10. General view of the orchestration ProcessBookOrders 


‘The orchestration receives a book order and checks whether the 
customer is a VIP. affirmative, the process sends a congratulation 
message. 

5. The first thing you have to do is, at project BookOrdersSolution, 
adda reference to the CAT Framework assembly that has been 
builtin the previous section. Add the Microsoft. BizTalk. CAT. 
BestPractices. Framework. dl] by navigating to the following 
location: C: press VChapterS VCATFranesorkaE T CATVS2015 
BizTalkCATInstrunentat ionFraneworkV1_40riginal \Microsoft. 
BizTalk. CAT. BestPractices. Sanples.Franenork\bin\Release 
Alternatively, you can add the CAT Framework Visual Studio 
project to the solution and add a by-project reference. 
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6, Add the BizTalk Server tracing component reference to the project. 
Tris installed into the GAC, and you can find a reference to it here: 
C:\Windows \Microsoft .NET\assenbly\GAC_MSIL\Wicrosoft. BizTalk. Tracing 


7. Gobackto the orchestration designer and, in the Orchestration, 
View Explorer, add a new variable called cal Token and specify 
System. Guid as the Type, as shown in Figure 5-11 





Figure 5-11. Declaring the callToken variable as System.Guid. 
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3. Just after the Activation Receive Shape, add a Expression shape 
and rename it initTrace, as shown in Figure 5-12. 





Figure 5-12. Insert the InitTrace expression shape 


9. Open the initTrace shape and type the following code to start a 


/itrace session starts into the orchestration 
//cal\Token will be used during the orchestration to generate 
events associated to that tracing session. 

callToken = Microsoft. BizTalk. CNT. BestPractices. 

Franework. Instrumentation. TraceManager.WorkflowConponent.. 
‘TraceIn("Orchestration starts"); 


10. Just before the End Orchestration shape, add a script shape and 
name it closeTrace, as shown in Figure 5-13, 





Figure 5-13. Insert the closeTrace expression shape 
11, Open the closeTrace shape and add the following code: 


/txace session is closed using the callToken 
Microsoft. BizTalk. CAT. BestPractices. Franeuork Instrumentation, 
Tracelanager. orkflowConponent. TraceQut(callToken, 
“orchestration End"); 
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12. Using the orchestration viewer, locate the Scope Hain and add 
two variables at the local scope level (see Figure 5-14) 
‘scopeNane: Type: System. String. Initial Value: Scope Main 
scopeStarted: Type: System. int64 Initial Value: 0 


Do not forget to add the : · characters to the value ofthe scopeNane 
variable! Ifyou miss this step, the build action wil fail. 





pa Man 


Figure 5-14. Creating scopeName and scopeStarted variables 


13. Rightafier the Scope Main starts insert a new script shape called 
‘TraceMainScopeStart. See Figure 5-15a. 


Tacit. 


Figure 5-15a. Creating the traceMain expression shape 


14. Open the TraceMainScopeStart and add the following code: 


/[Sending a new Starting Scope event to the trace. 


Microsoft. BizTalk. CAT. BestPractices Franework. Instrumentation. 


‘TraceManager .WorkflowConponent. TraceStartScape(scopeNane, 
callToken); 


15. Just before the Scope Nain finishes, insert a new script shape 
called TraceMainScopeClose, as shown in Figure 5-15b.. 


[le 
EE 


Figure 5-18b. Creating the traceMainClose expression shape 


E 
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16. Open the TraceMainScopeClose and add the following code: 


//Sending a new End Scope event to the trace. 
Microsoft. BizTalk. CAT. BestPractices.Franework Instrumentation. 
‘Traceanager .WorkflowConponent..TraceEndScope(scapeNane, 
scopeStarted, call Token); 


17. Now, on the left branch of the decide shape, insert a new script 
shape called TraceVipCustoner, as shown in Figure 5-16, 


— 











‘Congratulate 


] 
EU 





Tracevi. 


Figure 5-16. Inserting the traceVIPCustomer expression shape 


18. Add the following code to the TraceVipCustoner script shape: 


/[Send a nex event to show that the customer is a VIP Customer 
Microsoft. BizTalk. CAT. BestPractices.Franework.Instrumentation. 
‘TraceManager.WorkflowConponent.TraceInfo("Custoner has been 
detected as VIP); 


19. Now, on the right branch of the Decide shape, insert a new script 
shape called TraceNonVipCustoner. See Figure 5-17. 


ESI 
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aco 


Figure 5-17. Inserting the traceNanVIPCustomer expression shape 


20, Add the following code to the TraceNonVipCustoner script shape, 
as shown in Figure 5-1 





/[Send a new event to show that the customer is a not VIP Customer 
Microsoft. BizTalk. CAT, BestPractices. Framework. Instrumentation. 
‘Tracellanager.WorkflowComponent.TraceInfo("Custoner has been 
detected as non-VIP"); 
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Figure 5-18, ETW enabled orchestration view 


What would the output trace be if you run the application? 
As you can see, each expression shape within the orchestration creates a trace line in 

the ETL stack. In the next section, you see how to start and read traces, We are not ready 

yetto tes the solution because you need to learn how to control and examine the traces, 


Using TraceLog to Control Trace Sessions 


‘TraceL.og is a tool designed to start and stop trace sessions. By default, it is not included 
in any Windows installation, You can install the TraceLog tool from the Visual Studio 
Installation, as it is included as part of the Windows 10 SDK, as shown in Figure 5-19, 


E 
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og Visual Studio 





Figure 5-19. Installing the Windows app development tool 


A trace session isa period where a trace provider, such as BizTalk Server, is 
generating trace messages. The trace engine maintains a set of buffers for the trace 
session to store trace messages until they are flushed to a TraceLog or to trace 


There are three basie types of trace sessions: 


+ Non-real time- 





Trace messages directly in a log fle. This is the 
default behavior. 


+ Real-time—Instead of using a log le, real-time sessions send trace 





messages to a trace Consumer like DebugView, TraceView, or 
TraceFmt tools. 


^ Buffered—Keep messages in memory and does not send m 





sages 
10 any log file or consumer. This is very useful when trying to capture 
very long running issues as it runs in circular mode, overwriting 
messages when a maximum size is reached. 


With the 





CAT Framework, our custom BizTalk artifacts will act as providers logging 
new trace messages to a trace session, 
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Starting a New Circular Trace Session for BizTalk Server 


‘To start a new BizTalk Server trace session, you can use the following command in the 
‘Visual Studio command prompt environment: 


First, create a folder where you want to save traces. This book uses 
CrMiizTalkTraces.TraceLog.exe -cir 100 -start BizTalkServerTrace -flags 
OXTFFFEFFF -£ c:\BizTalkTraces\BizTalkServer.etl -guid #02316AFB,4148,42e4, 
BB7F,3AA92B96A98A -b 128 -nax 100 -rt 





To see real-time traces, open DebugVien.exe after running this traceLog.exe command. 





Note At this stage you do not have any BizTalk solution working with ETL traces, 
so attempting to run that command will generate nothing. Later, you will deploy the 
BookOrders solution and you could test it. 





Parameters explanation: 





. -cir NaxFileSize- Specifies circular logging in the event TraceLog 
fle. It expects a parameter called MaxFileSize that specifies the 
maximum size of the file in MB. In this case, we are using 100. 
Therefore, when the size ofthe TraceLog file reaches 1000 MB, new 
added trace messages will replace the oldest ones. TraceLog will be 
captured in this loop until you stop the session. 








+ Start SessionName—Starts a new trace session using the 
we eall 





Sessionlane parameter to identify the session. In this ca 
the session BizTalkServerTrace. 


+ -flags TraceLevel—Used to specify a trace level. At the moment of 
writing this book, the following trace levels are available: 


+ "none" set TraceLevel value =0x0 
+ “low” set TraceLevel. value =0x1 
+ tedium" set TraceLevel value -03 
+ "high" set Tracetevel value -0x7 


+ Tall" set TraceLevel value =0x7FFFFFFF 
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In this example, we are using OX7FFFFFFF as the trace level (all). Depending on the 
selected value, the engine will generate more or less data. 


+ -f LogFile—Used to provide a file in where all trace messages will 
be stored. In our ease, i's c:\BizTalkTraces \BizTalkServer. et]. 








+ -guid TraceComponentGuid—You can use this parameter to tell th 
engine to filter for a specific provider, The CAT Framework sends 
trace messages using the following providers: 

+ Pipelines: #691CB4CB, D20C, 408e, 8CFF, FDBAO1CD2F7S 
+ Workflow (Orchestrations): #D2316AFB,414B,42e4,B87F, 
3AA92B96A98A 


+ DataAccess (used internally} 
#2E5D6508, 72F9, 43¢9,477, 73366212895 


‘+ Transform (BizTalk maps): #2648548, 5AF3, 4dbe, 8602, 
73E98965378E 

+ Service (used intemally): #E67E8346, 90F1, 408b, AF40, 222B6E3C5ED6 

+ Rules (Business Rules): #7820466, 590F, 4991, 9287, 3F00BA62793D 

+ Tracking (BAM): #5CBD8BA0,60F8,401b, 8FF5,C7F3D5FABE41 

+ Custom (custom NET components): #6A223DEA, F806, 4523, BADO, 
3120CC4F63F9 

+ Inour case we are filtering for Orchestration providers: 2316AFB, 
4148,4264, BB7F ,3AA92B96A98A. 


+ -b BufferSize—Specifies the size, in KB, ofeach buffer allocated for 
the trace session. Ifyou do not pass this parameter, a default value 
will be generated based on system hardware resources, 

^o -nax NunberOf8uffers—Specifies the maximum number of 
buffers used, 

+ ort Starts a real-time trace session—This{s very important 
if we want to use DebugView to see the trace information in. 
real-time mode, 
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Stopping a BizTalk Server Trace Session. 
First you must fush the session using the following command: 


Tracelog.txe -flush BizTalkServerTrace 





When engin 
the TraceLog or trace consumer immediately (DebugView is an example ofa trace 


receives a flush request, events in the buffers are del 





red to 


consumer) 
‘As lush does not stop the providers, you need to call the -Stop command: 


Tracelog.exe -stop BizTalkServerTrace 


Examining the ETL File 
You can use the TraceLog format tool to convert the ETL file to fat file. 
‘tracefnt.exe TracelogFileNane -o OutputTraceLogrileNane -tmf DefaultTWrFile 


Unfortunately, at the moment of writing this book, the CAT team did not update 
the ETW Framework, and the tracefnt „exe version has a dependency in a DLL called 
‘traceprt.d11. Without itis not possible to format the output ETL file. You can find this 
DLL in the companion TracingTools folder located here: 
(C:\APRESS\Chapter5\CATFranework4ETW\CATVS2015\BizTalkCATIns trumentationF 
‘ameyorkV1_40riginal \TracingTools 
Therefore, change to this path when you want to format the ETL fle to TXT. 


Using the BizTalk CAT Instrumentation Controller Control 
Trace Sessions 


"The BizTalk CAT Instrumentation Framework Controller is an easy-to-use GUI designed 
Tor the BizTalk CAT Instrumentation 

allows you to start and stop a trace, adjust filter options, log toa fle, and/or enable real- 
time tracing to DebugView. You can download it from here: 








imework. The tool works asa ETW controller and 


https: //github. con/t fabrahan/BizTalkCATIFControlley. 
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You can install it from the companion folder: 
(C:\APRESS \Chapter5\CATFranenork4€TW\8izTalkCatInstrunentationController¥1_0_o 


Just run the BizTalkCATInstrunentationControllerSetup.asi fle and follow the 


Instructions. 

Once installed, start the tool from the Windows program menu (or from installation 
folder, by default at C:\Program Files (x86)\BizTalk CAT Instrumentation 
Framework Controller 1.0). The user interface is shown in Figure 5-20. 
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Figure 5-20. BizTalk Instrumentation Framework controller user interface 


"The first thing you should do is to set up the paths for DebugView and the Notepad 
text editor (consumers ofthe traces). You can do this by licking on the File menu and 
selecting Options, as shown in Figure 5-21 
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option: x 


Path to Dig ere 
APRESS Chaplet \DebusView\Dbgvew one Bome.. 














Patho pretened tent estor: 





D Lotoloto Debug ———— 











Figure 5-21. Configuring the trace consumers 


DebugView is included in the companion folder at C: \APRESS\Chapters\DebugView. 


Testing the BookOrdersSolution and Exploring ETL Traces 


In this section, you learn how to use the CAT Framework Controller to see the ETL traces 
generated by the BookOrders solution. 
Follow these steps; 


1. Remove the BookOrdersSolution using the BizTalk 
Administration console (Fit exists) 

2. Using the BizTalk Server administration console, import the 
BookOrdersSolution MSI file located here: 
C: PRESS Chapters VBookOrdersSolrtionVBookOrdersSolutionWSI 


3, Install the MSI, either by checking the "Run the Application. 
Installation Wizard to Install the Application at the Local 
Computer" option at Import MSI wizard (the previous step), or by 
double-clicking on the sms file from Windows Explorer 





4. Refresh the BizTalk Server administration console and start the 
BookOrdersSolution BizTalk application. 


Locate the CAT Framework Controller application using the 
Windows Start button, as shown in Figure 5-22. 
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Figure 5-22. Start the CAT Framework Controller 


6. Once the tool starts type the trace name. In this case, type 


BizTalktrace 


Choose the Select All option from the Detail Level dropdown box. 


3. Set the Trace Output by selecting Trace to a File and Trace to 


DebugView (real-time), 


9. Select Orchestration/Workllow Component forthe Trace Filter. 


10. Click the Start Trace button, The sereen should look ligue 
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Figure 5-23. Overview of the tool settings 
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1L. Click the Launch DebugView button to start the tool. At this stage, 
all he BizTalk Server FTL trace activity will be captured in real 
time by the DebugView tool. 

12. Nowit’s time to test the BookOrdersSolution. Using Windows 
Server Explorer, locate and execute the testing file: 


C: PRESS Chapters \BookOrdersSolution\BookOrdersSolution\Ports\ 


SendioVIPBookOrder. bat 


13. Observe the DebugView window, as shown in Figure 5-24. 
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Figure 5-24. Exploring the DebugView output. 


‘You can see how the tool shows the BizTalk Server trace information. 


1i. Using the CAT Framework Controller, click the Stop Trace button 
to finish the capture. 

15. Nowclickon the Open Log in Text Editor button to see the trace 
information using notepad. exe. 
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Instrumenting Using Business Activity Monitoring 


BAM can also be used to gather and monitor sensitive operations data related to the ow 
ofa business process, The following milestones and data are typically implemented for 
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this objective (among others): 


Busi 


Business Activity Monitoring (BAM) is a tool for monitoring and analyzing data, The data. 


is pres 





Starting and finishing times of orchestrations and custom pipeline 
components—By daing this, you can track the duration ofthe whole 
process. 

A point of time when orchestrations send and receive 
messages—This is remarkably interesting as you can gather th 
duration of your request-response and solicit-response process 
Although this information can be gathered if tracking events are. 
enabled, storing this information into the BAM databases could be 
of immense value when you are troubleshooting latency problems 
related to complex business scenarios that cannot be quick 
unraveled by querying the existing tracking information. 

















Error information—Whenever orchestrations pipelines, or custon 
components raise exceptions, you can get the exception code 
and description and attach it to the current activity definition, 
‘This technique can also be used to store errors related to business 


processes 
less Activity Monitoring 


ted in a real 





portal site 


Th 


BAM portal provides users with a web interface far viewing the data collected by. 


BAM, as shown in Figure 5-25. 


me view of business state presenting the information in the BAM 
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Figure 5-25. BAM portal view 


BAM exposes a NET API that can be used to gain visibility into data external to 
BizTalk processes, such as NET Windows Form applications, ASP.NET, Workflow 
Foundation, and more. At e moment of writing this book, the following interfaces were 
available: 
+ OES-OrchestrationtventStrean- Insert data into the BAM tables 
asynchronously, and as it participates in orchestration transactions, it 
should be called from an orchestration. 
+ DES:DirecttventStrean- This class inserts data synchronously. 
“This is typically used when real-time data is required, as it does not 
introduce latency. On the other hand, it could affect throughput, as it 
+ BES: BufferedEventStrean—Designed for high-throughput 
scenarios, it buffers the information and asynchronously inserts the 
data into the BAM databases, 


E 
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Although BizTalk pipelines do not implement formal BAM API classes, they expose 
the IPipelineContext interface that you can also use to insert dara into the BAM 
databases through the messaging event stream class implementation, 


BAM Components 
BAM includes the following components (se Figure 5-20) 


+ BAM interceptors—The collection of data (message context 
properties and message data) from orchestrations and pipelines is 
implemented as BAM interceptors. These interceptors monitor the 
data as itis being processed and collect information that has been 
identified, as necessary, The Tracking Profile Editor (TPE) is used to 
create these interceptors. Although developers can create custom 

‘ptrs by implementing WCF interceptor configuration files, if 

you use TPE, coding is not required. 








+ BizTalk Primary Import database—This is where interceptors save 
the essential data, This database contains stored procedures, tables, 
triggers, and views that are dynamically generated when deploying a 
BAM Definition, Data is kept in this database while the information 
is within defined Windows time and is then moved to the BAM 
Archiving database by the BAM Archiving Job, 





+ BAM activity aggregations and OLAP cubes—Developers and. 
business analysts can generate aggregations to present the 
information in a grouped way. These aggregations are maintalned 
represented by cubes within the SQL Server Analysis Service and 
generated by integration packages run under the control of the SQL 
Server Integration services, 
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Figure 5-26. BAM portal components view 


Designing the Instrumentation Activities for a BizTalk 
Server Solution 


"The first step to implement successful instrumentation using the BAM feature is to 
review the business process and design the BAM activity definitions based on that. 
You can use Table 5-1 for guidance. 


Creating the Book Orders Approvals Activity Definition 


In this step-by-step guide, you will create an activity that will generate all the elements 
detailed in the previous section (see Table 5-1). 


am 
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Table 5-1. Activity Definitions Example 








‘tem Name Description Type 

StartTine Time stamp when the process initiates. Milestone 

TransactionCode Code of the process. Gathered from the Text (max length 10) 
incoming message 

Transactionpate Date of the transaction. Gathered from the Text (max length 10) 
incoming message 

Transactionok True it the process completes without errors. — integer 
False if there was an error. 

Transactiontrror Eror description in case of a failure. Empty it Text (max length 255) 
mo errors found. 

CRMSoliticTimeStanp Time stamp to store when the CRM request Milestone 
happened. 

CRMRequestTimeStamp Time stamp to store when the CRM response Milestone 
arrived. 

EndTine Time stamp when the process finishes. Milestone 





Follow these steps: 
1L. Open the Excel BAM template located here: 
C:\Program Files (x86)\Microsoft BizTalk Server 2016\ExcelDir\Bam.xla 


2. Once Excel opens the sheet, go to the Add-ins toolbar option and 
select the BAM > BAM Activity option. 


3, The BAM activity wizard will load. 


4, On the Business Activity definition window, click the New Activity 
button, as shown in Figure 5-27. 
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Figure 5-27. Creating a new business activity definition 


5. Inthe new activity window, type BookOrdersApprovals and click 
on the New Item button, as shown in Figure 5-28. 
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Figure 5-28. Creating a new activity tem. 
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6. Setthe Item name to StartTine and sel 





the type item type. Then Click OK, See Figure 5-29, 
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Figure 5-29. Creating a new activity ltem. 


7. Repeat Step 6 for all of the items detailed in Table 5-2. 


Table 5-2. List of Elements to Define for the Activity 


Business Milestones as 





— [ee] = | 








‘tem Name Description we 

TransactionCode Code of the process. Gathered from the. Text (max length 10) 
incoming message 

TransactionDate Date of the transaction. Gathered from the Text (max length 10) 
incoming message 

Transactionük True ifthe process completes without errs. Integer 
False if there was an erro. 

Transactiontrror Eror description in case ot a failure. Empty it Text (max length 255) 


no errors foun. 


GRMSoliticTimeStanp Time stamp to store when the CRM request 


happened. 


CRMRequestTineStamp Time stamp to store when the CRM response 


EndTine 


arrived. 
Time stamp when the proces finishes. 


Milestone 


Milestone 


Milestone 





Figure 5-30. List ofthe items of the activi 


m 
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Once you finish adding the elements, your screen should look Figure 5-30. 











Click OK to close the New Activity dialog box. 
Click OK in the Activity Definition dialog box. 

Now that the activity definition has been created, ts time to 
create the BAM view that will be used to populate the BAM portal. 
“The wizard for view creation will oad the welcome sereen. 

Click Next. 


On the View Creation dialog box, select Create a New View and 
click Next. 

In the New View Creation window, type BookOrdersView 

as the view name and select the previously created activity, 
BookOrdersApprovals. See Figure 5-31. Click Next 
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Figure 5-31. Creating a new BAM view 


18, Inthe View Items dialog box, select All Items and click Next. 
See Figure 5-32. 





€ x 


Men B Ven Vern 


























Figure 5-32. Selecting all view items from the activity definition 


E 


n. 


i5. 


CHAPTERS INSTRUMENTNG BIZTALK SOLUTIONS 


‘Take a moment to review the Monitoring View Creation dialog 
box. You have three main option buttons here: 


+ New Alias You use this option to provide an alias to an 
item view. This is useful when the activity elements have a 
nondescriptive name. In this scenario, you will not create an alias 
as the activity elements are quite cleat 





+ New Duration —This option will enable you to create duration. 
fields in the view. You will use two milestones, and the BAM. 
feature will calculate the duration by subtracting both values 
"This is very useful if you want to know the duration of activities 

+ New Group—Vou can use a milestone group to set related 
milestones together; for example, the beginning and end 
milestones that define how long a book order is valid. 





Click on the New Duration button and set following properties 
(sce Figure 5-33): 


Duration Name: Orchestrationburation 
Start Business milestone: StartTine (SookOrderstpproval) 
End Business milestone: EndTine (SookOrdersapprovals) 
‘Time Resolution: Second 


Click OK, 
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Figure 5-33. Setting the duration properties 


16, Repeat Step 1510 create a duration name called CRHCa Duration: 
Duration Name: CRMCal1Duration 


Start Business milestone: CRASoLicitTineStanp 
(BookOrdersApproval) 


End Business milestone: CRHRequestTineStanp. 
(BookOrders Approvals) 


Time Resolution: Second 


17. On the NewBAM View:View tems window, click Next. In this 
scenario, you will not create dimensions and measures. This 
‘option is very interesting to create aggregation data in the BAM 
reports, 


18, Click Nest on the View Summary window. 
19, Click the Finish button to finish the process. 

20, The wizard wil show the Excel view ofthe activity. 

21. Goto the BAM menu and select the Export XML option. 


Choose a folder location and set the fle name to 
BookOrdersActivityDefinition.xnl. 
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Deploying the BAM Definition XML File 


BizTalk Server administrators and developers use the Deploy All BAM Management 
utility command to deploy a BAM definition from the XML definitions file exported from. 
the Excel sheet. 
‘The BAM utility creates the necessary elements within the SQL Server databases so 
the BAM feature can populate the business data associated with the activity definition, 
Follow these steps 





1. Open a command prompt as follows: Click Start, el 
cnd, and then click OK. 





ck Run, type 


2. Navigate to the tracking folder by typing C:\Program Files 
(486) Wicrosoft. BizTalk Server 2016\Tracking and atthe 
command prompt. Press Enter. 


3. Type the following instruction: 


br. EXE. deploy-all -DefinitionFile:"C:\APRESS\Chapters\, 
BookOrdersSolutionBAMMBookOrdersSolutionVBAMV 
BookOrdersActivityDefinition.xnl” 





4. Press Enter 


everything runs successfully, vou will see the screen in Figure 5-34. 
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Figure 5-34. Viewing the BAM.exe output 
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Now i you explore the BANPrinaryInport database, you will see that 
the BAMLexe tool created the tables shown in Figure 5-35. 
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Figure 5-35. Exploring the BAMPrimaryImport created tables 


As well as the views shown in Figure 5-36, 
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Figure 5-36. Exploring the BAMPrimaryImport created views 
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Configuring and Deploying the Tracking Profile 


Now that everything is ready from an infrastructure point of view, it's time to define how 


the activity tables are going o be populated. For that purpose, you will use the tracking 
Profe editor tool (TPE) that wil link orchestration interesting points and data to the 
BAM activity definition. 


Follow th 








se steps: 
Remove the BookOrdersSolution using the BizTalk 
Administration console (Fit exists) 


Using the BizTalk Server administration console, import the 
BookOrdersSolution MSI fle located here: 
C: MPRESS Chapters BookOrdersSolut ionBAM\BookOrdersSolution\nsi\ 








Install the MSI, either by checking Run the Application 
Installation Wizard To Install The Application At The Local 
Computer at the Import MSI wizard (previous step), or by double- 
clicking on the msi file from Windows Explorer. 


Refresh the BizTalk Server administration console and start the 
BookOrdersSolution BizTalk application. 
Locate and launch the Tracking Profile Editor tool (TPE) in the 
BizTalk Server 2016 Windows Start menu. 





Click on the link Click Here to Import a BAM Activity Definition, 
as shown in Figure 5-37. 
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Figure 5-37. Importing a BAM activity definition 


7. Inthe Import Activity Definition window, locate 
BookOrdersApprovals and click OK. See Figure 5-38. 
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Figure 5-38. Selecting the BookOrdersApprovals activity definition 
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3. TPE will now show all the item definitions created for the activity 
BookOrdersApprovals, as shown in Figure 5-39. 
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Figure 5-39. Exploring the BookOrdersApprovals activity 


9. Click on the Select an Event Source link, as shown in Figure 5-40 








Figure 5-40. Selecting an event source 
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10. Locate the orchestration BookOrdersSolution and click the Next 
button. See Figure 5-41. 
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Figure 5-41. Locating the BookOrdersSolution application as an event source 
1L. Select the orchestration BookOrdersSolution. 
orcProcessBookOrders and click the OK button. 


12, TPE will show you now the selected orchestration with the 
associated activity items, as shown in Figure 5-42, 
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Figure 5-42. Exploring the whole TPE profile 


Ws time to link the events of the orchestration to the activity definition. 
13. Drag and drop the receive shape Receive BookOrders to the 
StartTine milestone of the activity definition, as shown in. 
Figure 5-43, 


—E 

















Figure 5-43. Linking orchestration events to the activity definition milestones 
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14. Dothesame for the EndTine milestone (see Figure 5-44). 








| 
| 
| 
| 
= 


15. Do the same for the Request Response CRM activities, as shown in 
Figure 5-45 











Figure 5-44, Linking the send ToCRM send port to the activity definition milestone 


























Figure 5-48. Linking the CRM Request Response send port to the activity 
definition milestone 
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With these actions, the BAM feature will be able to track orchestration 
events to the BAM activity and insert the right data to the BAM. 
Primary Import tables. 

Ws time now to assign the data elements to the activity definition. 
16. Right-click the first receive shape of the orchestration and select 
the option Message Payload schema, as shown in Figure 5-46, 
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Figure5-46. Selecting the Incoming message data to link items to the activity definition 


TPE will show you the list of available elds. 


17. Now drag and drop the OrderTd and OrderDate fields 
corresponding to the activity definitions, as shown in Figure 5-47. 
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Figure 5-47. Linking orchestration data to the activity definition data milestones 
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18. Right-click the receive Approvals receive shape of the 
orchestration and select the Message Payload Schema option, as 
shown in Figure 5-48. 
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Figure 5-48. Selecting the incoming message data to link the approval data 





19, Nowlink the Approved element from the response schema to the 


activity definition, as shown in Figure 5-49. 
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Figure 5-49. Linking orchestration data to the activity definition data milestones 
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20. Save the tracking profile using the File menu. 

21. Goto the Tools menu and select the Apply Tracking Profile option 

22, TPE will inform you that tbe profile was successfully applied. 


"The binding between the business process and BAM is applied, 
which will make BAM events be triggered when the message 
arrives in the orchestration and populate the proper data in BAM 
Primary Import database, 


Now you can test the solution and check if the BAM primary 
import table for book orders contains the BAM data, as shown in 





Figure 5-50. Observing the associated BAM data 


Alternatively, you can log in to the BAM portal and perform a query 
in the activity search section, as shown in Figure 5-51. 
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Figure 5-81. Observing the associated BAM data using the BAM portal 


Instrumenting Creating Custom Performance Counters 


As most of the out-of-box BizTalk Server counters are at host lev 





it might be very 
Interesting to create your own performance counters from business or operational 
point of views, You could develop performance counters in the following BizTalk Server 
elements 


+ Custom pipeline components 
+ Custom components (BizTalk and external assemblies) 
+ Orchestations 

+ Custom adapters 


For example, ifyou create custom performance counters for a specific orchestration, 
counter data will be fulfilled only when that orchestration executes, Ideally, all 
performance counters should be reset when host instances start and stop. 

You could use these counter information later in performance reports. It will save 
youa huge amount of ime, Examples of useful performance counters are: 


+ Messages received by an orchestration, 


+ Messages sent by an orchestration 
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^ Number of ertors by orchestration 
+ Number of errors at pipeline level 


In addition, you could also use this technique to monitor your BizTalk Server 
platform because if those counters remain static for long period of time, that usually 
‘means that BizTalk Server is not processing messages (for that particular orchestration, 
application, of course), 


Developing Custom Performance Counters 


In this scenario, you have an orchestration called orcPracessBookOrders. 
that will receive book orders requests. This orchestration is bound to the host. 
BizTalkServerApplication and, therefore, will run under the context of 
BizTalkServerApplication host instance process. 

When the orchestration receives a new message, it will use the component 
PerfomanceCounterhelper in an attempt to create the following Performance Counter 
Category, BizTalkOrchestrations ProcessBookOrders, with these performance 








+ #Recelved Messages. 
+ Sent messages 


Ifthe category already exists, it will not be created. This is very useful because you 
do not have to create counters manually as the component will check it for you every 
time the BizTalk Server engine is creating a new orchestration instance (it implements a 
‘constructor to overwrite categories upon creation though). 

‘The component has an overload method for categories creation. If the orchestration 
is using side-by-side versioning, you have the option to exe 





the category with the 
following format: 


BizTalkOrchestrations VersionNunber_ProcessBookOrders 





Figure 5-52 shows that the component is creating a performance counter category 
called BizTalk Orchestrations 2.0 ProcessBookOrders. The number 2.0 identifies 
the orchestration version. In this way we could have counters associated to specific 


‘orchestration versions, 
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Figure 5-52. Observing the custom performance counters 


How Does It Work? 


When the o 





ration has been activated by a specific host instance, the component 
will attach both performance counters to the HostInstance process by creating a new 
performance counter instance called Bi zTa ServerApplication:BizTalk'sgBaxDb: 
BizTalk2016. This information is automatically gathered by the component using the 
following BizTalk Server WMI classes: 

+ MSBTS Orchestration.To get the HostName. 


+ NSBTS_Msp@oxSetting. To get the MsgBoxDb database name. 
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By picking up the current process server name. This is very important as the. 
‘Orchestration can potentially run in all of your servers. For example, if you have two 
BizTalk Servers, ServerA and ServerB, and the host for that orchestration has host 
instances running in all of them, counters will be generated locally in both servers by 
creating the following counter instances: 


+ BizTalkServerApplication:BizTalktsgBoxDb:ServerA 
+ BizTalkServerAppLication:BizTalkMsgBoxDb:ServerB 


‘You might be wondering why counters instances are attached to the BizTalk Server 
process. Well ifyou do not attach counters to a process, they will be gathering data 
until the server reboots and you will need to implement a mechanism to reset counter. 
instances values somehow. 

After testing the BizTalk Server application, you can see counter values by opening 
Windows perfmon tool, as shown in Figures 5-53 and 5-54. 
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Figure 5-53. Examining the performance output using perfmon tool 
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Figure 5-54 shows the report view 
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Figure 5-54. Examining the performance output in report format 


Book Orders Orchestration Flow Diagram 


First, the orchestration receives a book order message. As the message gets into the 
‘orchestration, perf helper component is called to increment the value of the #Received 
messages counter by one. Then the orchestration sends the message and after that, the. 
value of the #Sent messages counter is increased by one. See Figure 5-55. 
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DhTalkserverApplicstion host instance. 





orcProcessicokOrders 

































































Figure 5-55. Book orders orchestration diagram 


Examining PerformanceCounterHelper Component 


"This is the component that implements all the performance counter logl You can. 
find the Visual Studio solution in the companion code file at C: PRESS Chapters 
PerfornanceCounters \BaokOrdersSolut ion\PerfornanceCounterHelper. 
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Figure 5-56 outlines the class definition for the component. 


Ses 

© sDefautCategortime 

Methods 

© sdaalueToACounter 

© createCategony 

© PerformanceCounterteiperClss (+1 overload) 
© retumHostiastanceNameForProcess 


Figure 5-56. Examining the performance counter helper class diagram 


The createCategory Method 


"This method will create the Performance counter category within the Server 
performance counter repository, Users can force the recreation of categories by setting. 
the parameter bforce to True. Go through the code and read the comments as the code 
is self-explanatory. 


public bool createCategory(string slCategoryName, bool bforce = false) 
i 

/[xeturns true if the category can be created. This is just 
creating the category and the draft counters. 
Has Counters are declared as multiInstance, they will be 
created and assigned every time a BizTalk HostInstance 
process an orchestration of this type. 
/[35 force, the category will be deleted first and then recreated 
Boolean r = false; 





try 
{ 
if (bforce) //if user is forcing re-creation 
t 
Jie delete the category first 
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if (Pex formanceCounterCategory Exists (s1CategoryNane)) 
t 
PerfornancecountexCategory.Delete(slCategoryNane) 
Jine delete the category 


) 
if (IPerfornanceCounterCategory. xists (sICategoryane)) 
t 
//this is creating a collection of data counters. 
Mie will use it at the end of the If to create Counter 
metadata configuration associated with the category. 
CounterCreationDatatollection counters = new 
CountexCreationDataCollection(); 
Hive create the counter received nessages 
CounterCreationbata receivedhessages = new 
CounterCreationData(); 
receivedtessages.CounterNane = "i Received Messages" 
receivediessages.Counterielp = "Total number of 
received messages"; 
Teceivedhessages.Counterlype = PerforanceCounterType. 
NunberOFItens64; 
counters. Add(receivedtessages) ; 
‘Hue create the counter dsent messages 
CounterCreationData sendMessages = new 
CounterCreationdata() ; 
sendlessages.CounterNane = "f Sent Messages"; 
sendMessages.Countertelp = "Total number of sent 
messages 
sendHessages.CounterType = PerfornanceCounterType. 
Nunberü£tenst 








counters. dá sendiessages); 


(iCxeate a nen category with the new orchestration 
perfornancecounters 
(Me set up the counters as multilnstances 
(PerfornanceCounterCategoryType.ultiInstance) 
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[so we could add them later to the Host instance 
PerfornanceCounterCategory.Create(slCategoryVane, 
slCategoryNane +" counters", 
PerfornanceCounterCategoryType.Multilnstance, counters); 


) 
T = true; 
) 
catch (Exception) 
{ 
x= false; 
throw; 
} 
return t; 


} 


When this method completes, ifyou open the perfmon tool, you will see that the 
‘category BIzTalkOrchestrations ProcessBookOrders has been created, with their 
respective performance counters attached to it. See Figure 5-57. 
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Figure 5-57. Examining the expected output using perfmon tool 
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There isan override with the following definition: 


public bool createCategory(string slCategoryNane, string sVersion,bool 
bforce = false) 
"This override will add a version token to the category (if you are using 


side-by-side versioning, y 
‘orchestration. 





xu should have different performance counters per version of 





The addValueToACounter Method 


"This method performs the following actions: 
+ Creates counter instances 


+ Attaches that counter instance to the BizTalk process. By doing this, 


counter data will be reset when host instances stop. 








vents by 1 the value of count 





(received or sent messages). 
Here is the code: 


public bool addValueToACounter(string sCounterFanilyNane, string 
counterNane, string real0rchestrationNane) 
( 
bool r = false; 
string sCounterFullNane = 





try 
{ 
sCounterFullame = sCategoryName + sCounterFanilyNane; 
Systen.Diagnostics.PerformanceCounter p = new 
PerfornanceCounter(); 
p.CategoryNane = sCategoryNane; 
p.CounterNane = counterNane; 
[we build the instance nane. 
p-Instancelane = returnliostInstanceNaneForProcess (realOrche 
strationNane); 
//the following line will attach the counter ta the BizTalk 
Process. If you want the counters to be independent of the 
host instance, 
aos 
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Choose PexfermanceCounterInstancel.ifetine.Global. 
pInstancelifetine = PerfornanceCounterInstancelifetine. 
Process; 
p-Readünly = false; 
p-Increnent(); //ve increment the counter by 1. 
X = true; 
H 
catch (Exception ex) 
( 
x = false; 
throw ex; 
J 
return r; 


$ 


After the method execution 
Instances as well, as shown In Figure 5-58. 





the performance monitor tool will show the associated 
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Figure 5-58. Examining the instances of the custom performance counters 
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Consuming the Component in an Orchestration 


The orchestration orcProcessBookOrders receives orders and verifies ifthe customer is 





a VIP customer. If that is the case, the component creates the output message inserting a 
‘congratulations message. 
Follow these steps: 
1. Using Visual Studio Open the Solution BookOrdersSolution, 
located here: C: MPRESSVChapters \PerfornanceCounters\ 
BookOrdersSolution\BookOrdersSolution.s1n\, 





Once loaded, open the Orchestration orcProcessBookOrders. 


The orchestration has two main scripting shapes that call the PerformanceCounter 
Helper, as shown in Figure 5-59. 








Figure 5-59. Points where the component is called to populate the custom counters 
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3, Double-click the first script shape SetPerfCounters. See Figure 5-60. 


LI MEET 
Vener rece bts Coen 
— Mam Viren 


— — 
——— 


























Figure 5-60. Setting up the performance counter helper component. 

"The first line calls the default constructor for the PerfornanceCountertiel per to 
create the performance category with all of the associated counters. 

"The second one increments by 1 the value of the (Received Messages Counter, as the 
‘orchestration has just received the incoming message. 


4, Now open the Expression shape SetSentPerfCounter. 
See Figure 5-61. 
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Figure 5-61. Incrementing the sent messages counter to 1 
308 


CHAPTERS INSTRUMENTNG BIZTALK SOLUTIONS 


"This script block inerements by 1 the value of the #Sent Messages Counter since the 
orchestration has just sent the output message in the previous step. 





5. Using the File Explorer, navigate to the folder: C: \APRESS\ 
Chapters \Per fornanceCounters\BookOrdersSolution\Setup. 


6. Run the SetUp.cnd command file as an administrator This will 
deploy and test the solution by dropping 10 messages in the 
input folder 


7. Open perfmon and add the following performance counters. 
BizTalkOrchestrations ProcessBookOrders (see Figure 5-62) 





a #Recevied messages 


b. #Sent messages 




















— 








EE = 


p 


Figure 5-62. Adding the custom performance counters to a perfmon view 


d. Open the perfmon console and explore the results You should get 
something similar to Figure 5-63. 
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Figure 5-63. Viewing the populated data 


Summary 


In this chapter, you learned how to take advantage ofthe event traces for Windows 
to implement efficient traces for BizTalk Server. Although you could create your ows 
custom components to consume the BizTalk tracing assembly, this book showed you 
‘how to migrate the old BizTalk CAT Framework to the 4.6 framework version, thus you 
‘can use it with your BizTalk Server 2016 solutions and enable high performance traces in 
your projects, 

"This book assumes that you already have experience logging information to the 
Windows event log and extends that capability by using BAM to instrument business. 
and system important milestones. All of this can be enriched by creating custom 





performance counters that can populate business or system data. 
In the next chapter, you learn how to use development techniques to proactively 
Improve how BizTalk Server applications run in production. 
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CHAPTER 6 


Developing High- 
Performance Solutions 


In previous chapters, you learned how to optimize the BizTalk Server environment based 
on application priority levels and performance SLA definitions. Additionally, in Chapter 5, 
you learned how to instrument your BizTalk Server solutions using ETW, customer 

perfor 





nce counters, and BAM to enhance system monitoring, Now i’s ime to discuss 





the most common optimizations that you, as a developer, will need to know to develop. 
robust BizTalk S 
You can improve the performance ofa BizTalk Server solution by using the following 





er solutions. 


elements 
+ Schemas 
^o Orchestrations 
+ Maps 
+ Pipelines 


Improving Schema Definitions 


In this section, you learn how to improve the schema definitions. This topic is more 
important than it seems as BizTalk Server engine uses schema definitions to represent 
messages, and as you have learned previously, messages are the center ofall the 
information that goes through BizTalk Server. Follow the recommendations detailed 
in this section to simplify schemas because as the complexity increases, overall 

nce di 





perfor ses. This is especially true for large fles and unclear schemas with 
several optional elements and nodes, 
‘The idea for schema optimization isto reduce the time that the BizTalk Server engine 
takes to retrieve messages for further processing. 
Agno Bintan 2019 a 
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Length of the Element Names 


Ifyou ever worked with the swift adapter, EDI, or HL7 specifications, you probably 
noticed that the element names look like weird codes. Most of those schemas definitions 
has 





ea huge number of elements and nodes, and ifthey were using very descriptive 
names, the size of the final message instance would be very large. 

When you are developing a new solution, especially for applications that rank high 
‘on the application transaction levels, you should consider reducing the length of the 
name for the nodes and elements, because the final size of rhe message will be also 
decreased. This is especially true for the nodes that have repeating records or for very 
large messages. 





Note Keep in mind also that using deceptive names can complicate 
troubleshooting and developing, especially in maps, where non-specific names 
can become quite deceiving. Therefore, you need to find a balance between 
performance and clarity. Use common sense. For instance, if you are working with 
a schema that has an element name called CustomerCode, consider reducing 
itto CxId. If the root node name is called CustomerInformation, rename it 
Cxdata or CxInfo, 


Do not reach a point where maps or code become a nightmare to understand. 





Message Properties Performance Recommendations 


Whenever using promoted or distinguished properties, you should keep in mind the 
following performance factors and recommendations 





+ Reduce the number of written and promoted properties and. 
eliminate those that are not needed. 





+ Warning using XPath expressions. 
long primarily when the elen 
‘Therefore, the more distinguished flelds you have, the larger the 





Path expressions can be very 





Is located very deep in the message. 


context size. This situation affects the overall performance of the 
platform. Whenever possible, consi 
the beginning ofthe schema, 





moving the deep elements at 
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+ Property name length. In the same way that reducing the size 
ofthe element names also reduces the size of the message, itis 
recommended to reduce the length of the properties as much 
as possible, Shorter names ensure that the engine consumes 
less memory and still provides business functionality. This fact 
is especially true for distinguished fields as they do not have 
256-character limitation. 





+ Routing. Ifyou are not planning to use the property for routing 
or correlation, do not promote it! Just distinguish it, Promoted 
properties consume more resources as the engine inserts them into 
the subscriptions table, while distinguished properties are not. Also, 
ifyou enable property tracking, SQL Server process consumes more 
memory, processor, and IO resources, as it must insert the tracking 
information into the tracking data tables within the Message Box and 
then to the tracking database. These two facts might not be relevant 
‘while you are coding, but when the solution goes live and must. 
process millions of instances per day, believe me: it matters! 








+ Promoted property location. Especially for fla file scenarios, 
performance is affected by the position ofthe promoted property. 
‘within the schema definition. Promoted properties are found faster if 
you position them at the beginning of the schema, 


+ Extremely Low latency environments, If messages are small (fewer 
than 100 kilobytes), you can de-serialize the message into a NET 
class abject and access the publie static fields and properties (instead 
of using XPath). If the message needs complex business rules, 
accessing data using the properties exposed by an instance ofa NET. 
objectis faster than using XPath expressions because XPath loads the 
full message into the memory every tine it executes. 





"The following code shows an example ofa serialized message that exposes 
distinguished properties 


using System; 
using Microsoft. XLANGs. BaseTypes; 
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namespace NetClass 
{ 
[Serializable] 
public class MyBookNaneSpace. 
( 
public MyBook() 
t 
SBN = "101928818910111"; 
bookTitle = "BizTalk Server 2016 book"; 
} 


[DistinguishedFieldattribute()] 
public String iSBN; 


[DistinguishedrieldAttribute()] 
public int bookTitle; 


‘The code to create an instance of this object should be implemented within 
the context ofa construct shape because the properties are implementing the 
Distinguishedrield attribute (defined in Microsoft XLANGs.BaseTypes). The following 
code checks ifthe ISBN property equals "101928818910111" and, if yes, it changes the 
bookTitle property to "BizTalk 2016, performance tuning and optimization". 





msplessageIn = new MyBooktaneSpace.yBook(); 
3f (nsplessageIn.iSBNe» "101928818910111"} 
( 

msghessageIn.bookTitle ="BizTalk 2016, performance tuning and optimization" 


Canonical Schemas 


Canonical schemas are message definitions used to encapsulate the internal logic of 
your processes. The idea isto decouple the schema definitions of the integrated parties 
from the ones used by the BizTalk application, They ate extremely powerful, especially 
‘when receiving information from entirely different sources. 
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Asan example, imagine that you are developing a BizTalk application that 
‘consolidates customer data to a CRM application. Customer information can be received. 
from four different systems and it must be sent to three CRM solutions. As the solution 
requires advanced business process and granular exception handling, you develop an 
‘orchestration that will receive and send canonical messages. By implementing itin this 
way, you avoid the creation of one orchestration per source system. See Figure 6-1. 
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Figure 6-1. Canonical integration example. 











In the previous example, customer data is received through four different receive 
locations. Receive part executes then the corresponding map that transforms the 
incoming message into the canonical message used by the orchestration. After the 
business rules are executed within the orchestration, send port will transfa 
canonical message to the right destination system format. 

Benefits of canonical schemas include: 





the 


+ Schema changes from the source and destination systems will 
not affect orchestration received/sent messages, as these will use 
canonical schemas from receiving and sending Information. This wil 
reduce the number of deployments. 


+ New parties can be added to the solution without implementing. 


global changes at the orchestration level, by creating new receive 
location/send ports and developing new maps. 





+ Performance increases. As complexity of the solution is reduced, this 
approach can have a positive impact on performance (depending un. 
the number of instances). In the previous example, instead of having 
four dedicated orchestrations per source system, you develop only 
one that interacts directly with canonical messages, 
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Improving Orchestrations 


‘The following sections discuss 





related to orchestrations 





vant techniques and topi 
that you should consider at early stages ofthe project. 


To Orchestrate or Not To Orchestrate, That Is the Question 


Orchestrations are not always the best option for all integration scenarios because 
they consume more resources than message routing, You should consider the use of. 
orchestrations when you have any of the following requirements: 


+ Transaction support 
+ Granular exception handling 
+ Deby 





ration and rehydration to save resources 


+ Persistent points to recover from failures 





+ Correlation 
+ Convoy patterns 
+ Business activity monitoring at the business process level 


none of these techniques and features are required, then consider avoiding 
‘orchestrations and move business logic using a combination of custom pipeline 





‘components and ports. This technique diminishs 
database and consequently reduces latency. 


the overhead to the Message Box 


Using Orchestrations When Transaction Support Is Required 


BizTalk orchestrations offer a transactional programming model that includes support for 
recovery from failed transactions using compensation and exception handling, All arches- 
twations expose a property called Transaction Type that can have the following values: 
+ Alomie—Enables a transaction to automatically roll back to a 
previous state in case the transaction does not successfully complete, 
+ Long running—You use this type of transaction when the 


implemented business process can span longer time durations (y 
even), contain nested transactions, and use exception handling to 








recover from error scenarios. 
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+ None—Iyou just want to catch exceptions without transaction 
support, you should set the scope or the whole orchestration to None. 

You can set the transaction type property using the orchestration viewer and 

accessing the orchestration properties, as shown in Figure 6 





oreProcessBookOrders Orchestration Properties 
m 
Description 
Module Exportable Fae 
Namespace Boot OrdersSolstion 
jet Type Orchestration Properties 


Type Modifier 
Typename 








Figure 6-2. Available transaction types for orchestrations 


Using Orchestrations to Benefit from Dehydration 


Especially for applications where application priority levels or performance behavior 
rank higher, the use of hardware resources can increase exponentially. Orchestrations 
typically consume services that take some time to deliver the response. The BizTalk 
Server engine will nt ever know when the response is going to get back to the flow 
‘because what is happening under the hood of the service is completely out ofthe scope 
ofthe BizTalk Server engine, Now, Imagine that at some point an application has an 
‘orchestration that executes 1000 instances simultaneously. Ifthe total in-memory 
‘representation of that orchestration is 300 KB, the 1.000 instances will consume 
300.000 KB (292 MB) of the process memory. 
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While orchestrations are idle, ready to run or waiting for something to happen, the 
BizTalk Server engine will store the orchestration state to the Message Box database to 
save hardware resources related to the BizTalk Server machine. In the previous example, 
Ifall orchestrations wait for the response at the same time, 292 MB of memory will be 
released as all orchestration states are stored into the Message Box database. 

‘When orchestrations get activated again, the orchestration loads the orchestra 
state from the latest point and resumes the flow of the orchestration, This process is 
formally called rehydration. 











Note Dehydration may take place whenever the engine estimates that an 
orchestration is idle for an extended period. The only exception to this rule is 
‘when the flow of an orchestration enters an atomic scope. Actions taken within 
an atomic scope do to initiate a dehydration situation because these types of 
transactions persist the orchestration state only once, at the end of the scope. 





While the orchestration engine oversees the dehydration feature, you can control its 
behavior by changing the orchestration throttling settings at the host level 


Orchestration Throttling Settings Related to Dehydration at the Host Level 


‘You can control the dehydration behavior of orchestrations by accessing the settings 
shown in Figure 6-3 using the BizTalk Administration console atthe host level 
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Figure 6-3. Accessing orchestrations thrattling settings at the host level 


Dehydration behavior: 


+ Never—This option is disabling the dehydration feature for all the 
orchestrations running under that specific host. That means that 
when an orchestration is consuming a service or waiting for another. 
event to occur, the orchestration engine will keep the orchestration 
into memory. This option will alleviate SQL Server CPU, memory and 
10, but puts strain on memory consumption. 


. Always—Dehydration will occur always. While this option will save 
memory resources to the BizTalk machine, it can potentially cause an 
overhead to the SQL Server hosting the Message Box database. 


‘+ Custom —Dehydration will take place based on the Time Based or 
Subscriptions thresholds. 
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Time Based: 


+ Maximum threshold—Idle orchestrations will be held into the host 
instance memory for a maximum number of seconds specified 
by this setting, After that, the orchestration engine forces the 
dehydration of the orchestration itis still idle). The default value is 
1800 seconds (30 minutes). 

. Minimum threshold — Orchestrations are considered for del 
when they are idle for atleast the number of seconds specified in this 
threshold, 





ration, 


For ex 





npl 
orchestrations ofthat host are completed in less than one second, the orchestration 
engine will never dehydrate those orchestrations, On the other hand, ifsome 
orchestration instances are idle for more than one second, the orchestration engine 
considers those orchestrations for dehydration. 

If, for some reason, there are orchestration instances taking longer than the 
‘maximum threshold, the orchestration engine will force dehydration if orchestrations 
are idle. 

Subscriptions: 


you set the minimum threshold to one second, and all the 


+ Pause at—Orchestrations have subscriptions to messages that are 
stored to the main host queue tables. When the number of messages 
in this queue (associated with the orchestration subscription) is 
equal or higher that the Pause at threshold, the messages are not. 
delivered to orchestrations instances until the number of messages in 
the host queue decreases to the Resume At threshold. 





^ Resume at—As explained in the Pause At setting, this threshold 
controls whether the message engine starts to deliver new message 
instances to orchestrations. 


For example, if you set the Pause At threshold to 1.000, the message engine stops 
delivering messages to the orchestrations, running under that host, when the number of 
associated messages to that orchestration, reaches 1.000. 

the resume atis set the Resume At threshold to 600, the message engine will 
resume delivering messages to those orchestrations when the number of pending. 
messages decreases to 600 or below. 
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Reducing the Impact of Persisted Data 


BizTalk Server engine stores the orchestration state within the Message Bax in the 
following situations: 


+ End ofa transactional scope (atomic or long running) 
«o Atdebugging breakpoints 

^o Ata Start Orchestration shape 

«o Atthe Send shape except in an atomie transaction) 

. When an orchestration is suspended 

^o When the system shuts down in a controlled manner 
+ When dehydration occurs 

. When an orchestration completes 


‘Therefore, reducing the number of persiste 





points is all about making the right use 
‘of transactional scopes and send shapes because the rest of situations are out of reach 
(the engine controls orchestration events that cannot be modified). 





Reduce the Number of Persistent Points in Exception handling 


Ina section of your orchestration you want to catch exceptions, but compensation or 
timeout control is not required, then set the transaction type of that scope to none. This 
technique will eliminate a persistent point from the whole process as the BizTalk Server 
engine will not save the state ofthe orchestration when that scope finishes. 


Reduce the Number of Persistent Points in Sending Operations 


Ian orchestration needs to send multiple messages using a send shape per operation, 
ary these approaches instead: 
+ Encapsulate all the send operations with a send port group when the 
outgoing message is the same one. With this action the engine will 
persist the information only or 





because you are using only one 
send shape. 
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‘+ Ifthe first option is not possible, then try to wrap up all the send. 
shapes within an atomic scope because atomic scopes generate only 
one persistent point at the end of the scope. 


For instance, imagine that you have an orchestration that has to send three messages 
to three different destination systems, as shown in Figure 6-4 










[o 
erg 


ec 


Figure 6-4. Orchestration with three persistent points 


Myou develop the orchestration by adding three subsequent send shapes, the 
orchestration engine will persist the orchestration state three imes. Ifthe application 
ranks high in the application priority levels for the number of transactions, the amount 
‘of persisted data per second will be large. 

Therefore, as discussed, you can decrease the number of persistent points by 
wrapping up all the send shapes within an atomic scope, as shown in Figure 6-5, 
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Figure 6-5. Orchestration with one persistent point using an atomic scope. 
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Size of Persisted Data 


Try to keep th 
messages at the most optimum level possible. In BizTalk Server you can declare variables. 


e size of an orchestration as small as possible by declaring varlables and. 





and messages at orchestration or at scope level, as shown in Figure 66. 


— Properties 
Orcheemtin Parte 


(B mstpprovalRequet 
Bl mustppronltespane 
H roseo 


y Variables 
—— 


Menit 
—— 





Figure 6-6. Declaration of variable and messages at different levels 


Variables and messages declared at orchestration level are always stored in the 
Message Box in each persistent point, while the ones declared at scope level will be only 
stored ifthe scope has not been completed yet. This small action will reduce the time 
spent by the host instance when storing and retrieving persisted data from the Message 
Box database. This is essential for all BizTalk Server applications, but it is especially 
crucial for solutions where application priority levels rank higher for the transaction. 








levels 
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For instance, imagine that an orchestration needs to process 100 transactions per 
second and it is using a message of 10 KB size that is declared at orchestration level. 

Making some quick calculations, you get the following data: 100 KB * 100 = 10.000 KB 
per second to store that message per orchestration persistent point. 

Ite orchestration has 10 persistent points, you get a total size of 100.000 KB 
(around 97 MB) of persisted information, and only for that message! 

Declaring that message at scope level, if only two persistent points are occurring at 
that scope, that will reduce the size of the persisted data to 20% because now, only tivo 
persistent points out of 10 will store that message. 











Improving Orchestration Latency 


"The following sections discuss the most common techniques to reduce orchestration, 
latency. 


Loading and Unloading Assemblies Into Memory 


into the memory associated to. 
s. The first request can take up to 15 seconds to start processing 





When host instances start they load all the assemb 





the hostinstance proe 
messages. Ifan orchestration is idle for a long time (ready to run or dehydrated) or The 
BizTalk host instances has not process a new orchestration instance for more than 30 

minutes, assemblies will be unloaded from the host instance until a new request arrives, 

You can control the way host instances load and unload assemblies by adding a 
configuration setting to the BizTalk configuration file that is called AppDonain. 

"This setting allows you to define profiles with shared settings so that you can assign. 
BizTalk Server assemblies to load with those profile configuration settings. Il you do not 
specify custom profiles, in other words f you do not want specifie settings per BizTalk 
Server assembly, there is a default profile called Default Spec that will be used to load all 
the configuration, 











<AppDonains AssenbliesPerDonain ="10"> 
dDefaultSpec SecondsIdlebeforeshutdown: 
SecondsEnptyBeforeShutdown="3800"> 
</AppDonaiins 
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+ Seconds dleteforeShutdown—BizTalk Server host instances will 
unload all the assemblies in this domain when all orchestrations are 
idle for a period af time specified by the SecondsTdleBeforeShutdown 
setting. Idle orchestrations are the ones that have the status as ready 
torun or dehydrated. Specify -1 to when a domain should never 
unload when idle. 


+ SecondsEnptyBeforeShutdawn—BizTalk Server host instances will 
unload all the assemblies in this domain when itis empty for a period 
of time specified by the SecondsEnptyBeforeShutdown setting. This 
is the number of seconds that an app domain is empty (does not. 
contain any orchestration instances) before being unloaded, Setting 
itto -1 will cause the AppDonain to never unload, even when i's 
empty. 





In the previous example, SecondsTdleBeforeShutdown is set to -1. Therefore, 
BizTalk host instances will not unload assemblies that have idle orchestrations. 
SecondstnptyBeforeShutdonn is set to 1800, and that adomain gets 
empty, the BizTalk Server engine will wait30 minutes to unload all the assemblies 
within the host instance and it will not load them until a new orchestration instance is 


sans that whi 





activated. 





Note When an idle but non-empty domain is unloaded, all the contained 
orchestration instances are dehydrated first. 





Ifyou want to create a custom profile based on specific assemblies, with different 
‘configuration than the default one, you have to add the AppDonainSpec section to the 
AppDosainspecs section. 

For instance, imagine that you have a mission-critical application called 
BookOrdersApp] ication and you want the BizTalk host instance processes to not unload 
the BookOrders application assemblies (even when they are empty for an extended 
period). 
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First, you have to create the custom AppDosain by adding the AppDonainSpec section. 
to the AppDomainSpecs section: 


<AppDonainSpecs> 
<AppDonainSpec Nane="DonainForBookOrdersAppplication” 
SecondsTéleBeforeShutdoun-"-1" SecondstnptyBeforeShutdoun-' 
«/AppbonainSpec» 

«</AppDonainSpecs> 





And now you just need to tell the BizTalk Server engine which assemblies should be 
‘considered as part of the custom domain “DonainForBookOrdersAppplication’ To do 
that, you need to create an ExactAssignnentRules and include all the assemblies for the 
custom domain: 





«bactAssignnentRules» 
<ExactAssignmentRule AssesblyNane-"BookOrdersSolution, 
Version«1.0.0.0, Culturesneutral, PublickeyToken=9c7731c5584592ad" 

Assenblyane Q»""" AppDonainNane=" 

DonainForBookOrderstppplication -"^ />AppDonainiane 1" /> 


Replacing Send Shapes by Custom Code 


jor levels 





Especially for applications where application transaction or performance behi 
rank higher, consider removing the send shapes and implementing the send operation. 
using a custom NET component. 

‘This technique will reduce latency of the orchestration, because th 
evades the BizTalk messaging engine, and provides the following benefits 





ustom code 





. Iwil reduce the number of round trips to the Message Box database, 


‘+ Asyouare eliminating a send shape, the orchestration will have a 
persistent point less, 


Howew 





by implementing a custom send operation, you are not accessing the 


Tollowing BizTalk Server features: 





‘+ Pipeline features such as recovery interchange, message validation, 
disassembling and decoding/encoding, party resolution 





^ Sendportretries 
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+ Send port scheduling options 
+ Correlation 

. Backup transports 

+ Tracking 


+ Integrated monitoring and troubleshooting using the BizTalk 
administration console 


Creating Static Classes 


Whenever you are developing a custom NET component to provide additional functionality 
to orchestrations, consider creating them as static because they provide the following, 
benefis: 
+ Hardware resources Since statie classes have no Internal state, the host 
instance will reduce hardware resources such as memory and CPU. 


^o Decreasing the orchestration state size—This is very important because 
the orchestration engine will not have to save the status of the 
component on every persistent point or whenever it needs to dehydrate. 


+ Avoiding atomie scopes- Since components that require serialization 
must me placed under an atomic scope, NET components 
exposing stati classes or methods will not require to imple 
the serializable attribute and can be consumed everywhere in the 
orchestration without the need of using an atomic scope. This will 
reduce the number of persistent points per orchestration because 
atomic scopes generate a persistent point at the end of the scope, 





ient 


To create a class or method as stat 





you just need to add the static modifier to the 
declaration. The following example outlines how to achieve this: 


public statie class BizTalkServerHelper 
f 
public statie string GetConnectionstring(string connectionString) 
t 


return  conectionstring; 
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public static GetProperty (string itesProperty) 
{ 


return _itenProperty; 


Controlling Orchestration Memory Consumption 


Especially for large messages scenarios where orchestrations consume large amounts of. 


memory, you might have the need of tuning the default throtling behavior of 





consumption at the host instance level 


You can access these settings using the BizTalk Administration console at the host 


Instance level, as shown in Figure 6-7. 











Figure 6-7. Accessing orchestrations throttling settings at the host instance level 


as 
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As you can see, there are two sections-— Physical and Virtual. The first one refers 
thresholds related to physical memory consumption while the second one refers to 
virtual process memory consumption. Both sections have Optimal and Maximal usage 
thresholds that behave the same way, but specifically dedicated for the type of memory 
that they are related to. The threshold explanation is as follows: 


+ Optimal usage—When the consumed physical/virtual memory for 
that host instance is below this threshold, the engine does not throttle 
orchestrations. On the other hand, iit reaches this threshold, the 

stration engine will start throttling orchestrations instances to 





save physical/virtual memory resources. 


‘+ Maximal usage—Ifthe consumed physical/virtual memory reaches. 
this value, the throtdling condition becomes very aggressive and the 
host instance will not load new orchestrations until the consumed 

‘optimal usage. Maximum. 

ier than the optimal usage threshold. 


memory decreases to a value close to u 
usage should be equal or gr 








For instance, imagine that you have one BizTalk Server with 16 GB of available 
memory. 

You are developing a high throughput application that process a 5 MB file within an 
orchestration. Because you are creating two messages of that schema type to perform 
transformations and temporary operations, BizTalk Server loads 10 MB into memory per 
orchestration instance, 

Additionally, you are using four XPath expressions to access the content of specific 
elements within the message. As XPath expressions load the fll message into memory, 
you must add another 20 MB per archestration instance, 

Now, imagine that the number of concurrent orchestrations could potentially reach 
1400. 

Performing quick calculation, you get: 400 orchestration instances * (10 MB 
message size + 20MG of XPath expressions) = 12,000 MB (12 GB) of memory used by the 
hostinstance In the orchestration scenario. 





Note For simplification reasons, calculations do not include any extra data 
related to the orchestration itself such as variables, objects, and intrinsic 
orchestration state. 
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As the server has 16 GB, a consumption of 12 GB means a 75%. In this scenario the 
orchestration engine will start throttling orchestrations because the consumed memory 
exceeds the default 70% for the Optimal Usage threshold 


Reducing Orchestration Complexity 


As orchestration complexity increases, performance decreases, as most likely the 
following elements will also increase 


+ Number of shapes that the orchestration executes 
^o Number of persistent points 

«o Number of messages and variables 

+ Dehydration cycles 

+ Consumed memory 

+ Access 10 1/0 (network and disk) 

+ CPU resources 


‘Try to design business processes using a modular style by using the same concept 
‘of eanonieal schemas but applied to orchestrations. Create orchestrations that you can. 
reuse for several scenarios or try to segment orchestrations to isolate functionalities into 
different orchestrations, 

For example, imagine that you have developed several Web API services that exposes 
business functionality through REST. You have an orchestration that is calling those 
REST services seven times during the low. To consume those REST services within the 
orchestration, you have inserted seven request/response ports. The solution works fine, 
but the orchestration looks huge and complex. Deployment is also complicated as you 
must bind seven extra logical ports to physical ports. This is a perfect situation to isolate 
the REST service call in a separated orchestration, The new orchestration will use the 
canonical schema technique, so that changes at REST services level will not affect the 
business orchestration. The business orchestration will call the nested orchestration, 
providing the name of the service and the operation to execute, and it will do that 
sending and receiving canonical messages through the child orchestration. 
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Improving Maps Execution 


Adjusting the temp folder location and using the Xs1ConpiledTrans farm class will speed 
up map processing, especially if a fast disk is deploy 








din the local server. 





BizTalk Host Instance Temp Folder Location 


BizTalk Server engine uses the hast instance account temp folder to stream fles to disk 
‘when maps are executed, You should consider placing the temp folder in a separated 
disk to reduce the chances of disk contention. This is especially true when maps are 
‘executed under the following conditions: 


+ Processing large messages or complex schemas, 


+ Complex maps with hundreds of links or more. 


Using the XsICompiledTransform Class 


BizTalk 2016 introduces the use of the Xs]ConpiledTrans form class. This class is used to 
‘compile and execute XSLT transformations. In most cases, the Xs1Conpi ledTrans orn 
‘lass significantly outperforms the XsITransforn class in terms of the time needed to 
‘execute the same XSLT against the same inbound XML document. 

There is only one performance limitation with this new class: Because the XSLT is 
‘compiled to MSIL, the first time the transform is executed, there isa performance delay 
because the map need to be compiled first. However, following executions are much 
faster than using Xs]Transform, 

You can decide whether to use the XslConpi ledTrans form class by accessing the 
map property called Use XSL Transform, as shown in Figure 6-8, 
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Figure 6-8. Examining the Use XSL Transform property 


When you create a new map, the property is set to Undefined by default (same as 
true). That means that the map will be executed using the old non-compiled version. 
you want to use the compiled version, you must set the property to false, 











Note The recommendation is that you test the application with both settings and 
choose the one that gives you the best performance. 





Improving Pipelines 


"There ate several techniques to improve custom pipeline processing, but probably the 
memory management. In this section, you 
learn about using passthru pipelines, disposing objects efficiently, and how to avoid 
loading the full message into memory using streaming technology. 


‘most relevant ones are related to effici 
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Use the PassThru Pipelines 


The default passthru pipelines for receiving and sending do not use any pipeline 
‘component in the pipeline stages. Because of that, they just treat the message as a chunk 
‘of data. Therefore, if your solution does not require accessing the data of the message, 
you can set up the PassThruReceive pipeline. 

‘The same concept can be used in the sending layer. Ifthe message to be sent 
is already prepared for the destination system, you should consider setting up th 
PassThruTransmit pipeline. In this scenario, message validation will be done by the 
destination system, not by the sending layer. 











Disposing of Objects 


Although this is a general best practice for any kind of development, this is especially 
iue for pipelines that are part of applications that rank higher in the transaction 
application priority levels. You should consider disposing your objects such as helpers, 
database connections, or any other unmanaged resources. Failure to do this could cause 
a memory leak in the server, preventing BizTalk Server host instances from accessing 
memory resources that could have been released before, 

In your NET custom components, you can provide explicit control by implementing 
the IDisposable interface and override the declaration by adding custom code that will 
release external resources such as database connections, network drives, and so on. 
Then, you can call this method to make sure everything is released. 

BizTalk Server message engine uses the concept of resource tracker to add to 
it references of objects that must be released at a specifie moment of time. Custom 
pipeline components can add objects to this tracker through an instance of an. 
IPipelineContext object and then the engine will release them just when the pipeline 
‘execution has completed successfully, 

For Instance, in the Execute method of a custom pipeline component (explained 
deeply in further sections), you can load the message into a memory stream object and 
then add this instance to the resource tracker so that the message engine will 
it when the pipeline completes 





minate 
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public TeaseMessage Execute(IPipelineContext pContext, TBaseMessage pInMsg) 
I 

MenoryStream messageData = new MemoryStrean(); 


/1code to load the content of the message into the menoryStrean 


[lie add the memory stream to the resource so it gets disposed by the 
message engine. 


pContext.ResourceTracker.AddResource(nessageData); 


rest of the code 


return outHessage; 


) 
Avoid Using XMLDocument Objects 


Frequently, once a custom pipeline component recelves a me application has 





ssage, 





a requirement to read the receiving message to act based on the content by creating 
context properties, validate the message or any custom activity. In such scenarios you 
should avoid loading the message into an YALDocunent abject because it will load the full 
message into memory and its representation will cost up to 10 times the original data, 
Instead of using the XH Docunent, consider creating an XMLTextReader that can 
retrieve data from one of the following classes exposed by the Microsoft BizTalk. 
Streaming d11 assembly: 
+ Virtualstream 
+ ReadOnlySeekableStream 
+ SeekAbleReadonlyStream 
You can find an implementation ofthese techniques inthe SDK installation folder of 
BizTalk Server: 
C:\Program Files (x86)\Microsoft BizTalk Server 2016\SDK\Sanples\ 
Pipelines\ArbitraryXPathPropertyHandler. 
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‘An explanation ofthe scenario is detailed here: 
https: //docs.nicrosoft. com/en-us/biztalk/core/arbitrary-xpath-property- 
handler-biztalk-server-sanple?redirectedfron-MsDN 


Developing Pipeline Components to Improve 
Performance 


When orchestrations are not required or can be avoided, consider implementing 
advance processing at the pipeline level. This will improve performance as there will not 
‘be added lat 
hardware resources). 

Pipeline components are used in the pipeline phases to perform the following tasks: 





icy due to the orchestration engine (persistent points, dehydration, and. 


+ In receive pipelines—Decode, disassemble, validate, and party resolution. 


ind encode, 





* Dn send pipelines— Pre-assemble, assemble, 





Custom pipeline components can be dragged and dropped to pipeline stages using. 
the Visual Studio toolbox. By default, BizTalk Server includes the pipeline components. 
outlined in Figure 6-9. 





Search Toolbox. P: 
4 BizTalk Pipeline Components 
ko Pointer 


Wh Di Assembler 
Dh ASLencasee 

E) 
IML assembler 

Fiat fle assembler 

JSON decoder 

JSON encoder 

MIME/SMIME encoder 

XML validator 

ResendManager 

EnablDyramicPorRoutng 


lal EA l m 


Figure 6-9, Examining the BizTalk pipeline components in the toolbox 
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You can create custom pipeline components to provide extended capabilities 
that are not provided by all default pipelines. For example, you might have business 
requirements to: 


+ Receive messages that are zipped. 
+ Send zipped messages. 


+ Look fora database setting and insert it into the message context 
before the message gets published to the Message Box database. 


+ Improve the performance of the application by implementing the 
orchestration functionality within the pipeline. 


"This section guides you through the process of custom pipeline component creation. 


Types of Pipeline Components 


You can develop the following pipeline components: 


^ General Designed to get one message (no batch), execute relevant 
code, and return to BizTalk Server. See Figure 6-10. 








Pipeline 


|__ B 


[E 

















Figure 6-10. General custom pipeline component 
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+ Assembling—Designed to receive several messages and send one 
message to the engine. See Figure 6-11. 








Pipeline 


Th- 



































Figure 6-11. Assembling custom pipeline component 


+ Disassembling— Designed to receive several messages and send one 
message to the engine. See Figure 6-12. 








Pipeline. 


- 






































Figure 6-12. Disassembling custom pipeline component. 
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Custom Pipeline Interfaces 


At the pipeline stage, the BizTalk Server engine executes several specific methods 
to access the data stream that is going through the pipeline. Thus, your pipeline 
‘components should implement those methods so that the BizTalk Server engine ean 
receive or send messages. The way to provide those methods is to implement several 
interfaces that will help you send and receive messages to the BizTalk Messaging Engine. 
Depending on the type of component you want to create (general, assembling, or 


disassembling), 





ou need to implement different interfaces. 


Available interfaces 


TBaseConponent- Provides standard information to the BizTalk 
Engine and to the designer like Description, Name, and Version. It 
is shared across the three types of components (general, assembling 
and disassembling) 


‘Component —This interface is implemented only in general pipeline 
components, Itis exposing a method called Execute that is used to: 


+ Receive messages from the engine. 
+ Send processed messages back to the engine. 
IPipelineContext—Implements all document processing related 


Interfaces by providing the following main functionalities (among 
others) 


^o Retrieves pipeline and stage configuration settings. 


+ Provides the mechanism to interpret XSD schemas and BizTalk 
Server annotations. 


TPropertyBag--Used to store custom settings for the pipeline. 
Developers will use them to regulate how a pipeline process 
documents. Itis exposes two methods: 


+ Read, which reads a custom setting. 
‘+ Write, which saves a custom setting 


IPersistPropertyBag—Provides a mechanism to persist the 
properties identified by the IpropertyBag interface. 
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IDisassenblerComponent—Used only in disassembling custom 

components and can be associated only with receive pipelines, It will 

receive only one document and will produce one or more messages. 

Itis the typical scenario in which a batch message is received and 

contains one or more child messages with the same structure. It 

exposes two methods: 

+ Disassenble—It will disassemble the incoming message. 

+ GetNext—The disassemble method will create a set of messages. 
You need call the GetNext method to loop through them. When 


this method returns null, that means that you have reached the 
end ofthe set 





TAssenblerCosponent.— Used only in assembling custom 
components, Therefore, it can be used only in sending pipelines. 

it will receive multiple messages and will generate only one output. 
message. It exposes two methods: 


+ MddDocunent— Creates a lest of messages that will be included in 
the process of creating the individual output batching file. 


^o Assenble—Takes all the messages included in the previous list 
and creates the batch message, 


TDocunentSpec-Used to perform actions to the message content, 
such as 


Writing properties into the message context. 
+ Removing properties from the message context. 
+ Changing the message body part 

«o Adding or changing the target naneSpace property. 


‘ComponentUI—Used to display the icon in the Visual Studio toolbax 
and to validate configuration properties. 


YCustosTypeDescriptor— Used to provide user friendly names and 
descriptions of properties used in pipelines, 
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Attributes of Pipeline Components 


All custom pipeline components can behave in many ways. You can define custom. 
attributes to specify how the pipeline should be interpreted by the BizTalk Server engine 
and use itin the right pipeline stage, 

Here, you can see all the available attributes: 








CATID_Any—Specifies a component with no specifie category. Like 
general components 


CATID_AssenblingSerializer—The component is an assembling 
serializer 


CATID_Decoder—The component isa decoder. 


CATID Disassenblingearser— The component is a disassembling 
parser 


CATID_Encoder—The component isan encoder 
CATID_Parser—The component isa parse. 
CATID_PartyResolver—The component isa party resolver. 
CATID_PipelineConponent—he component isa pipeline. 
CATID_Receiver—the components a receive. 

CHTID Serializer— Ihe component is a serializer. 

CHTID Streanex— The component is a streamer: 


CATID_Transnitter—The component is a transmitter. 


Developing a General Custom Pipeline Component 


Asan example on how to use pipeline components to avoid orchestration scenarios, in 
this section you are going to learn how to develop a general custom pipeline component 
1o the message context. The application uses. 
that promoted property later for a message routing scenario, 

Figure 6-13 outlines what will you achieve. 


that writes a custom promoted property 
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Figure 6-13. Understanding the scenario 


Solution Overview 
"The solution will be implemented by developing three BizTalk Server projects: 


+The BizTalk project that will contain the schema used for property 
promotion 


+ The Custom pipeline component 


^ The Testing BizTalk project 


Developing the Property Schema Project 


"This project will contain a schema fle that implements the promoted property. 
Follow these steps: 
1. Using Visual Studio, create a new BizTalk project called. 
BookPronotedPropertiesSchena. 
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2. Addanewschema file called BaokPronatedProperties, 

3. Rename the main root name to PronatedProperties 

4. Goto the property schema file, Examine the Target Namespace 
property. You use this schema to promote the properties lat 
in the pipeline code, https: //BookPronotedProperties. 
PropertySchena 





Add the following Child Field elements: 
+ BookTrackld 


+ BookSystenld 


6. Right-click the BookTrackId element and select Quick Promotion. 





With this action, the BizTalk Editor creates the PropertySchena 
be used to store all af the promoted properties. 





schema that wi 


7. Right-click the BookSystenId element and select Shows 
Promotions. 


3. Select the Property Fields tab. 
9, In the Promote Properties dialog box, select BookSystenID and 

click on the Add button. Click OK. 

10, Choose Save AIL 

1L. Sign the output assembly by accessing the project properties. 
Right-click at the Project level and choose to deploy. The 
schemas will be deployed on the default application. 
Using Visual Studio, create a new BizTalk project called 
BookPronatedPropertiesSchema. 








Developing the Custom Pipeline Component Project 


n this section, you create a fully functional general custom pipeline component that will 


‘be used to write the properties created previously to the message context. 


"The idea is that the pipeline will expose the properties outlined in Figure 6-14 in the 


pipeline user interface. 
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* DET e 
iure 14. Examining the custom pipeline properties 


So that whenever a message goes through the pipeline, the BookTrackId property 
Will be filled with the custom PropertyValue 1234. 


Creating the Visual Studio Project for the Custom Pipeline 
Component 


All get 





al custom pipelines components should implement the following interfaces: 
+ GBaseCosponent 

«o Component 

^o GConponentiT 


+ GPersistPropertyhag 
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Follow these steps: 


1. Open Visual Studio as an administrator and create a new Visual 
Studio project. 


From the Templates menu, choose Visual Cf, select Class Library, 
and navigate to the location where you want to save it 


3. Type PropertyGeneratorPipeline in the Name and click OK. 
4, Rename the default CS class file to PropertyGeneratorPipeline.cs. 


Add the following references to the project as they will be used by 

the BizTalk Server Engine: 

+ Microsoft BizTalk.Pipeline, located in C:\Program Files 
(x86) Microsoft BizTalk Server 2016\Microsoft BizTalk. 
Pipeline.dll 


+ Microsoft XLANG. BaseTypes, located in C:\Program Files 
(x86) Microsoft BizTalk Server 2016\Microsoft XLANGS. 
BaseTypes.dll 


+ Microsoft XLANG, BizTalk Engine, located in C:\Program 
Files (x86)\Wicrosoft BizTalk Server 2016\Microsoft. 
XUNG BizTalk. Engine. dll 

^ Microsoft KLANG, BizTalk. ProcessInterface, located in C:\ 
Program Files (x86)\Microsoft BizTalk Server 2016\, 
Microsoft. XLANGs BizTalk. ProcessInterface.dIl 


+ Microsoft. XLANG. Engine, located in C:\Program Files (x86)\ 
Microsoft BizTalk Server 2016\Wicrasoft.XLANGs.Engine.dl1 
+ Systen.Drauing, located in C:\Program Files (x86) \Reference 
Assenb] ies Microsoft \Franework\.NETFramework\v4.5\ 
Systen.Drauing.dll 
6. Once you've added these, set the Copy Local property to True for 
all references but System. Drawing (it should be done by default) 


7. Goto the project settings and sign the assembly with a new strong. 
name key file. 
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Go to the Application category under the project settings and 
change the target Framework to point to NET Framework 4.8. 


In the Application Category, click the Assemblylnfarmation button. 
Enable the Make Assembly COM Visible setting, 
Save all your changes, 


Open the PropertyGeneratorPipeline file and add the following 
using statements to the namespace: 


namespace PropertyGeneratorPipeline 


using Systen; 
using System. Resources; 

using System.Drawing; 

using System. Collections; 

using Systen.Reflection; 

using Systen. ConponentModel; 

using Systen. Text; 

using Systen.I0; 

using Microsoft. BizTalk.Nessage.Interop; 
using Microsoft.BizTalk.Conponent. Interop; 


Now it's time to tell to the BizTalk engine that our component. 
isageneral custom pipeline component. You do that by adding 
the ConponentCategory attribute and setting it to CATID 
PipelineConponent so that it can be used in the Validate stage of 
the pipeline (i could act as a validator). 


‘Type the following code just before the 
PropertyGeneratorPipeline class definition: 


Jithe component is a pipeline 
[Conponentcategory(CategoryTypes.CATID PipelineConponent)] 
{ithe component is a general component 
[ConponentCategory(CategoryTypes.CATID Any)] 

/[3t can be used in the validate stage at the pipeline component. 
[ConponentCategory(CategoryTypes.CATID Validate) ] 
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15. Navigate to the PropertyGeneratorPipeline class definition and 
insert the following private variable declarations: 


public class PropertyGeneratorPipeline 
{ 
private string custonPronatedPropertyNane = null; 
private string custonProsotedPropertylameSpace = null; 
private string custonPronatedPropertyValue = null; 


‘You use these variables to save the local properties defined in the pipeline user 
interface. 





Namespaces Warning! As with every NET application, custom pipeline 
components elements (methods, properties, interfaces, and so on) are accessible 
through namespaces. It is very important that you set up the namespaces properly. 
If you do not, you cannot add the pipeline component to Visual Studio toolbox and 
you will get the following error 

"You have selected an invalid pipeline component assembly. 
Please check security settings for the assembly if you are 
loading it from an UNC path" 


To prevent this issue, you must check that your class namespace specification is 
the same as the project nameSpace. 





In this scenario, the class namespace is: 





namespace Microsoft.BizTalk.Pipelines.CustonComponent. 
PropertyGeneratorPipeline 


{ 


Go to the project settings and verify that the default namespace property is 
Identical. 
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Adding a Resource File 


‘You use resources to save the literal strings you use in the pipeline user interface and to 
store pipeline properties (such as Namespace, Name, and Value). See Figure 6-15. 





m CET o 


Figure 6-18. Examining the custom pipeline properties 


Follow these steps: 


L. In Visual Studio, right-click at the project level, choose Select. 
New ltem, Navigate to the Visual C# Items, and go to the General 
section, Select Resource File and type the following name: 
PropertyGeneratorResourceFile.rex 


2, Add the following string resources: 
a. Containers for user settings: 
+ custonPronotedPropertyNaneProp 
+ custonPronotedPropertyNaneSpaceProp 
«o CustonPronotedPropertyValueProp 
E 
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Leave the three of them with empty values, as they will be filed in later by the user 
using the pipeline user interface 


b. Containers to show the caption that the pipeline interlace shows: 


+ custonPronatedPropertyNaneSpaceText. Value: Custom 
namespace 


+ custonPronatedPropertyNaneText. Value: Name of the 
property to promote 


+ custonPronatedPropertyValueText. Value: Value of the 
promoted Property 


3. Now you are going to create the icon that will be shown in the 
‘Visual Studio toolbox. Click the Add Resource menu, go to New 
Image, and select BMP Image, as shown in Figure 6-16. 
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Figure 6-16. Adding a new image to a resource file 
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‘The Image designer tool will now load. Create your own icon; be creative! 





4. Gobackto the PropertyGeneratorPipeline class and below the 
local variables declarations, add a resource manager object 10 
access the pipeline design properties: 


private string custonPranotedPropertyNane = null; 
private string custonPronotedPropertyNaneSpace = null; 
private string custonPronotedPropertyValue = null; 


static ResourceManager resManager = new 
ResourceManager ("Microsoft BizTalk. Pipelines. CustonConponent. 
PropertyGeneratorPipeline", Assenbly.GetExecut ingAssenbly()); 
[system.Runtine. InteropServices .Guid("48BEC85A-20EE-s0ad- 
BEDO-319859A00DBC")] 





Note You should paste the GUID generated in the GetC1assID method of the 
Interface IPersistPropertyBag. Leave it like this for the moment, as it will be 
discussed again when the time comes, 





Adding the MsgDescriptionClass.cs 


This already created class provides the required functionality to deal with Resoures 
Manager properties, As it Is not BizTalk Ser 
Tor reference. 

You can visit https://nsdn microsoft .con/en-us/Library/systen.resources. 
esourcenanager(v-vs. 110) .aspx to get more Information about Resource Manager 





related, it has been included in the book. 





implementations. 


1. Right-click the Visual Studio Project and choose Add Exiting Item. 
Navigate to Module 6 source code folder and locate the class file 
called MsgDescription.cs. 


2, Choose Save All. 
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Implementing the custom Interface BaseCustomTypeDescriptor 


The MsgbescriptionClass.cs class defines a custom interface called 
BaseCustonTypeDescriptor. It contains all the logie to access the Resource Manager 


Information and retrieve and set properti 





in the resource file. In this section, you 


enable this implementation in the PropertyGeneratorPipeline class so that the custom. 


pipelin 
1 


component can access the user interface properties 


Add the implement BaseCustonTypeDescriptor tothe the 
PropertyGeneratorPipeline class 


public class PropertyGeneratorPipeline : BaseCustonTypeDescriptor 


Select the BaseCustonTypeDescráptor statement and auto- 
generate the constructor by selecting the Implement Interface 
Code Generator. 


Notice that Visual Studio will generate all the implementation, 
code for us. It is recommended that you use regions, so the code 
will be clearer later. Change the constructor and adapt it to the 
following, assigning the already created reshanager object: 


public PropertyGeneratorPipeline() : base(resManager) 
{ 
) 


Lets define now all the properties that will help us to Interact with 
the user defined properties in the pipeline: 





Add the implementation for the PropertyNane property 


[msgPropertyNane("“custonPronatedPropertyNaneProp"), 
Nsgbescription(" custonPronatedPropertyNanePropText”)] 
public string PropertyNane 


get ( return custosPronotedPropertyane; ) 
set { custosPronotedPropertylane = value; ) 


am 
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4, Add the implementation far the NaneSpace property 


I 
MsgPropertyNane(" custonPromotedPropertyNameSpaceProp"), 
Nsghescription("custonPronotedPropertyllaneSpaceText") 
1 
public string NaneSpace 
t 
get { return custonPronotedPropertyaneSpace; } 
set { custonPronotedPropertyNamespace = value; ) 
) 


5. Add the implementation for the PropertyValue property. 


L MsgPropertyNane( "custonPronctedPropertyValueProp"), 
Nsgescription("custonPronotedPropertyValueText ") 


1 
public string PropertyValue 
I 
get ( return custonPronotedPropertyalue; ) 
set ( custonPronotedPropertalue = value; } 
* 


Implementing the IBaseComponent Interface 


"The IBaseCosponent interface provides standard information to the BizTalk Engine and 
to the designer like Description, Name, and Version. I is shared across the three types of 
components (general, assembling, and disassembling) 

Follow these steps: 





1. Add the implement TBaseComponent to the class 
PropertyGeneratorPipeline: 
public class PropertyGeneratorPipeline : IBaseConponent 
2. Select the IBaseConponent statement and auto-generate the 
implementation by selecting the Implement Interface Code 
Generator. See Figure 6-17. 
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public class PropertyGeneratorPipeline: PESSE 


{£ [e 
b Implement interface. (^ 


Implement interface explicitly 


Figure 6-17. Implementing the interface declaration 


3. Notice that Visual Studio will generate all the implementation 
code for us. In this case, we will be using a region called region. 
TBaseCosponent. 


Let's write the code to implement all the interface properties. 
A. Navigate to the property Name and type the following code: 
get { return "Property Generator Component"; } 


"This property will be used to show the name ofthe component in the Visual Studio 
toolbox. 


5. Navigate to the Description property and type the following code: 
get { return "This pipeline will insert a custom promoted property"; } 


"This property will be used to show the description ofthe component in the Visual 
Studio toolbox. 


6. Navigate to the Version property and type the following code 
get { return "0.99"; ) 


"This property will be used to show the version of the component in the Visual Studio 
toolbox. 


7. Choose Save Al, 
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Implementing the Component Interface 


This interface is implemented only in general pipeline components. Itis exposing a 
method called Execute that is used to 


+ Receive messages from the engine 
+ Send processed messages back to the engine 
Follow these steps: 
L Add the Microsoft. BizTalk.Conponent.Interop.lCosponent 
interface. We need the full BizTalk Server namespace notation 
here because ifnot, Visual Studio will interpret we are trying to 
use the Systen.Conpanentadel .TComponent interface and not 
the one provided by BizTalk Server. 
public class PropertyGeneratorPipeline : IBaseComponent, 


Microsoft BizTalk. Conponent.Interop.IConponent 
I 


2. As you did far the IBaseConponent interface, auto-generate the 
implementation. Notice that Visual Studio will generate all he 





entation code for us, Iti re 





impl ommended that you use 

regions, so the code will be clearer ater In this case, we will be 
using a region called #region ICosponent. 

3. Navigate to the method InitNew and delete all the generated code 
inside the method. 


public void InitNew() 
{ 
} 





Note If you do not delete the code, the pipeline properties will not be populated 
because, by default, the code is throwing an exception 
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A. Navigate to the Execute method and type the following code: 
ty 


IBaseMessageContext msgContext = plnitsg.Context; 
msgContext krite(custonPronotedProperty 

Nane, custosPronotedPropertyNameSpace, 
custonPronotedPropertyValue) ; 


rnsgContext.Promote(custonPronotedPropert 
yNane, custonProsotedPropertyNaneSpace, 
custonPronotedPropertwalue) ; 
) 
catch (Exception ex) 
í 
if (pInsg t= null) 
t 
plntsg.SetErrorinfo(ex); 
) 
throw ex; 


) 
return pInisp; 


‘The Execute method will be called by the BizTalk Engine to get the full incoming. 
message and publish it later to the Message Box database, As we want to write a new 
promoted property to the context, we have fist to get the context itself. The object 
TBaseMessageContext has two methods to interact with the message context: 


^ Wirite—Used to write properties into the context, 


+ Pronote—Used to promote a property into the context, Basically itis 
changing the type from non-promoted to promoted. 


Both methods are expecting the following parameters: 


^ NameSpace—Namespace in which the property will be writen to the 
context (custoaPronotedPropertyNaneSpace) 


+ Nane—Name ofthe promoted property (custonPronotedPropertyNane) 
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+ Value—Value of the promoted property 
(custonPronotedPropertyValue) 


In this pipeline, these properties are assigned by the user using 
the pipeline user interface, 





5. Choose Save All 


Implementing the IPersistPropertyBag Interface 


"This interface is used to store custom settings for the pipeline. Developers use them to 


regulate how a pipeline processes documents. I is exposes two methods: 





^o Read, which reads a custom setting. 
+ Write, which saves a custom setting. 
Follow these steps: 


L Add the Microsoft. BizTalk.Conponent.Interop. 
TersistPropertyBag interface. We need the full BizTalk Server 
namespace notation here because if nor, Visual Studio will 
Interpret we are trying to use the System. Conponentodel . 
TPersistPropertyBag interface. 


public class PropertyGeneratorPipeline : IBaseComponent, 
Microsoft. BizTalk. Conponent. Interop.IConponent, 
Microsoft. BizTalk.Conponent. Interop. IPersistropertyBag 


2. Aswedidforthe IBaseConponent Interface, aulo-generate 
the implementation, Notice that Visual Studio will generate all 
the implementation code for us. It is recommended that you use 
regions, so th 
usinga region called #region IPersistPropertyBag. 





code will be clearer later. In this case, we will be 


3, We need no to create two methods for writing and reading values 
from the properties assigned to the pipeline (in the user interface) 

A, Create a new static method in the region IPersistPropertyBag 
called ReadPropertyBag and type the following code: 
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private static object ReadPropertyBag (Microsoft BizTalk.Cosponent. 
Interop. IPropertyBag pb, string propane) 
D 

object val = null; 

ty 

( 

ph.Read(propane, out val, 0); 
) 


catch (ArgunentException) 
t 
return val; 
) 
catch (Exception ex) 
1 
‘throw new Applicationtxception(ex.Nessage); 
) 
return val; 


) 
Create a new statie method in the region IPersistPropertyBag 
called WritePropertyBag and type the following code: 


private static void VritebropertyBg(Microsoft. izTalk.Conponent. 
Interop. PropertyBag pb, string propane, object val) 


{ 
ty 
( 
phHrite(propline, ref val); 
) 
catch (Exception ex) 
( 
‘throw new Applicationtxception(ex. Message); 
) 
) 


‘HAPTER 6 DEVELOPING HIGH-PERFORMANCE SOLUTIONS 
6. Navigate to the GetClass I method and type the following code: 
ClassID = new Systen.Guid("48BECB5A-20EE-40ad-BFDO-319859A00DBC") ; 


Generate à new GUID with the Visual Studio and paste it there. This GUID will be 
referenced at the beginning of the class definition. 

The method GetClassT0 will return the class ID of the component for usage from 
unmanaged code. 


7. Navigate to the Load method and type the following code: 





string val = (string)ReadPropertyBag(propertyBag, "Namespace 
if (val I= null) NaneSpace = val; 





val = (string)ReadPropertyBag(propertyBag, 
3f (val Ie null) PropertyNane = val; 


'PropertyNane"); 


val = (string)ReadPropertyBag(propertyBag, "Propertyvalue"); 
3f (val t= null) Propertyvalue = val; 


"This method reads all property bags from the store. The BizTalk Engine will call the 
Load method in two phases: 


+ When the pipeline initializes 
+ When the pipeline reads the data from a specific pipeline instance 
3. Navigate to the Save method and type the following code: 


object val = (object) custonPronotedPropertyNaneSpace; 
WritePropertySag(propertyBag, "Namespace", val); 


val = (abject)custonPronotederopertyNane; 
ritePropertyBag(propertyBag, "PropertyNane’ 





» val) 


val = (abject)custonPronotedPropertyNanespace: 
WritePropertyBag(propertyBag, "PropertyValue 





val); 





‘This method writes all values to the property bag’s store. The administration console 
will invoke the Save method when the user changes the values in the pipeline user 
Interface. 
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Implementing the IComponentUl Interface 


"This interface is used to display the icon in the Visual Studio toolbox and to validate 
configuration properties. 
Follow these steps: 





1. Add the IComponentU Interface. 


public class PropertyGeneratorPipeline : IBaseComponent, 
Nicrosoft.BizTalk.Conponent.Interop.IConponent, 
Microsoft. BizTalk. Conponent.Interop.1PersistPropertyBag, 
IConponentül 


As we did for the IBaseConponent Interface, auto-generate the 
Implementation. Notice that Visual Studio will generate all the 
implementation code far us. Itis recommended that you use 
regions, so the code will be clearer ater In this case, we will be 
usinga region called region IComponentUT, 





3. Navigate to the Validate method and type the following code: 


IEnumerator enumerator = null; 
ArrayList strList = new ArrayList(); 


åf ((custonPronotedPropertykameSpace != null) 8& 
(custoePronotedPropertyNaneSpace.Length >= 256)) 
( 

strList.Add("Imvalid Nase Space 





) 


if ((custonPromotedPropertyNane != null) 88 
(custonPronotedPropertyNane. Length >= 100)) 
{ 

strList.Add("Invalid property Name"); 


if ((custonPronotedPropertyValue |= null) && 
(custonProsotedPropertyValue.Length >= 256)) 
I 


ans 
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strList Add ("Invalid property valu 





) 
if (strlist.Count > 0) 
( 
enumerator = strList.GetEnunerater(); 
) 


return enumerator; 


"This method will validate that the pipeline properties are not null and have the 
appropriate size. IE not, the user will get validation messages. 


Installing the Custom Pipeline Component 
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Deploy the assembly into the global assembly cache. Use the 
Gacutil /T tool or any other preferred method. 

Copy the output assembly to the BizTalk Server pipelines folder, 
located by default in: 





:\Progran Files (x86)\Microsoft BizTalk Server 2016\ 
Pipeline Components 

BizTalk Server uses this folder to load all the pipeline components. 
Using Visual Studio, right-click over the toolbox and select the 


Choose Items option. Visual studio will show the component. 
loader tool. 


Select BizTalk Pipeline Components tab and click on the Browse 
button, as shown in Figure 6-18. 
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Figure 6-18, Adding the custom pipeline component to the toolbox 


5. Navigate to C:\Program Files (x86) Microsoft BizTalk 
Server 2016 Pipeline Conponents and select the 
PropertyGeneratorPipeline.dll file, 


Ieverything went well, you should see now your brand new 
pipeline component available In the toolbox when designing new 
pipelines. See Figure 6-19, 
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Figure 6-19. Adding the custom pipeline component to the toolbox 


Creating the Test Routing Application 
Follow these steps to create a test routing application: 


1. Using Visual Studio, create a new BizTalk Server project and call it 
CustonPipelineTesting. 


2, Addio the projecta new receive pipeline. 


3, Visual Studio will open the Pipeline Designer, as shown in 
Figure 6-20. 
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Figure 6-20. Examining the pipeline designer 
4. Now, using the toolbox, drag and drop the new custom pipeline 
component into the validate stage of the pipeline and choose Save ALL 
5. Adda new Schema file and call TestSchena xsd 
6, Rename the main root name Book. 
7. Add the following two field elements: 
+ Is 
+ Tue 
4. Choose Save Al 
3. Sign the assembly using the Project properties. 
10, Deploy the solution. 
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12, 


1 


n. 


is. 
16. 


n. 


18, 
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Jgnore all warnings. 
is time now to create the receive port. 

Open to the BizTalk Administration Console, go to BizTalk 
Application 1, right-click at Receive port level, and select add new 
One-way Receive Port, 

Under the receive port properties, go to the receive locations 
section and add a new receive location, 

‘Type the desired name and select the adapter type to FILE. 
Clickon the Configure button and set the folder you want BizTalk 
to receive messages from. 

Go to Receive pipeline and choose our custonpipelineTesting 
pipeline from the dropbox. 





Cliekon the ellipsis button and set the following values for dh 
custom properties 


^o Namespace: https: //BookProsotedProperties., 
PropertySchena 


«o PropertyName: BookTrackld 
+ PropertyValue: 1234 
+ Itshould look similar to Figure 6-21. 
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Figure 6-21. Examining the pipeline properties 


19. 


Now we will create Send ports, Using the BizTalk Administration 
Console, go to the Send ports section of BizTalk Server Application 1. 


Create a new static one-way send port 
Rename it SendPortALL 


Select File as the adapter type and configure it to point to the 
folder you want to Send All messages. 


Go to the Filters section and create à new filter. This send port will 
send all messages received by the receive port you have previously 
configured. Create this filter expression: 


BTS.ReceivePortNane = ReceivePorti (Go to step 19 and 
verify the port nane) 


Click OK. 


Create a new static one-way send port. 


aes 
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E 


Rename it SendPort1234. 
Select File as the adapter type and configure it to point to the 

folder you want to send messages with BaokTrackID = 1234. 

Goto Filters section and create a new filter, This send port will 
send all messages received by the receive port you have previously 
configured. Create this filter expression: 
BookPronotedProperties.PrapertySchena.8ockTrackId = 1234 
Click OK. 

Restart all host instances. This step is very important. Since we 


have deployed a new pipeline, BizTalk host instances need to load. 
the new assemblies into the memory again. 





Start the application and drop valid Book XML file instances on 
the receiving folder (generate a new instance using the Generate 
Instance tool in BizTalk Editor). The output send folders will have 
all outed files, 


Summary 


In this chapter you learned how to improve your BizTalk Server solutions from the basic 
building blocks of the schemas to maps, orchestrations, and pipelines. 


In general, you should develop your solutions to avoid or reduce the complexi 





‘of the orchestrations involved. This is when the use of custom pipeline components 
‘can boost your solution. In this scenario, pipelines can become crucial, especially for 
applications that rank higher in the application priority levels. 


Inthe next chapter, you learn two techniques that will enable you to reduce 


application downtime, 
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Decreasing Downtime 


In the previous chapter, you have learned how to develop solutions to improve schemas 
definitions, orchestrations, and pipelines, 

Another important factor of developing robust BizTalk Server solutions is to reduce 
application downtime. If the application bas a high availablity level (HAL) of, high 
availability is required. You will have to think ofa strategy to provide high availabili 
with zero downtime (especially for long-running processes that cannot be stopped). 

"This chapter covers two techniques that help you reduce the downtime of BizTalk 
Serv 














applications: 
+ Side-by-side versioning 
+ Business rules. 


Although security ls nota performance requirement, you will also learn how 


to secure data using the Enterprise Single Sing-On database to protect application. 
configuration settings in a secured store. 





Side-by-Side Versioning 


Most BizTalk Server applications provide functionality to mission-critical solutions 
that must run 24x7 with no downtime. The complexity of these applications is usually 
tremendous, as they interact with lots of different components and could potentially be 
involved in long-running processes, that can remain active for days, months, or even 
years. As you can guess, the deployment process of such applications becomes a very 





delicate task, since you cannot stop them while they are processing instances. 
Frequently, BizTalk developers and administrators deploy applications using the 

BizTalk Administration Console or PowerShell scripts. If new installation binary files 

include changes to orchestrations that are already running and processing instances, 

the BizTalk administration console will not update the orchestration until the person 

2 Agustin Minas 2018 = 
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performing the change terminates/suspends active instances. The same will happen 
to pipelines and schemas. Because of this, it is very important to develop a robust 
deployment procedure. 





How Can You Solve this Problem? 


There isa technique in NET Framework called side-by-side versioning and it applies 
to any kind of NET assembly. Luckily for us, all BizTalk Server Solutions create NET. 
assemblies, which means BizTalk Server is fully compatible with this technique, at the 
application level only. 

Side-by-side execution is the ability to run multiple versions of an application on. 
the same computer. You can have multiple versions of the CLR, and multiple versions of 
applications that use a specifie version of the runtime, all on the same computer at the 
same time, See Figure 7-1. 





Server A 
































Figure 7-1. CLR side-by-side execution diagram. 


The NET Framework extensively uses this feature o allow a server to run different 
versions of the CLR engine. In most scenarios today, servers can run different NET 
Framework versions, such as 2.0, 3.5, 4.0, and 4.6x, and consumer applications decide 
‘which engine the version will use by pointing to the right Global Assembly Cache stare. 
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Note BizTalk Server 2016 does not support side-by-side execution at the CLR 
Engine level, as Microsoft has tested the product in 4.6x versions of the NET 
Framework only. 





Component Version Side-by-Side Execution 


‘This is the when BizTalk Server uses its full potential. Since the BizTalk Server. 
leverages the NET Framework to execute applications at runtime, it uses assemblies to 
‘encapsulate all BizTalk artifacts. Therefore, you could have different assembly versions 
Where orchestrations, pipelines, custom code, and whatever you want to include, also 
have different versions. 

In the diagram shown in Figure 7-2, Application 1 is using component Version 1 
‘only. Applications 2 and 3 are using the same component but with Version 2, That will 
allow BizTalk Server to simultaneously run diferent artifact versions within the same 
BizTalk Server application. The idea Is that you will be adding new functionalities to 
the new version, while the old version willbe sill up and running and processing old 
version flows. 





Server A 








+ 
‘Conponent ‘Component 
version 1 version 2 









































Figure 7-2. Component side-by-side execution diagram 
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And there is more! By applying versioning to your BizTalk solutions, you can deploy 
side-by-side without having to stop the old version. Therefore, old subscribers will be 
still processing instances. It is recommended though that you create a new host for all 
the new deployments because you will not have to restart the old host instances. As soon 
asthe old systems are no longer using the old version, you can un-deploy the previous 
version of the orchestration. 





Applying Side-by-Side Versioning to a BizTalk 
Server Project 
In this section you learn how to apply side-by-side versioning to the book orders 
solution 
Follow these steps 
1. Open File Explorer and go to C: PRESS Chapter] SidebySideV 


2, Copy and paste the BookOrdersSolution to folder the same 
location and rename it to BookOrdersSolutionVersiont 





Note In an enterprise environment, instead of performing this action, you will label 
your solution in TFS as version 1. As this book is not covering source code control 
technologies, you are simulating labeling by creating a copy of the original folder. 





3. In File Explorer, run the following setup command: C: \APRESS\ 
Chapter7\SideBySide\BookOrdersSolution\Setup\SetUp.cnd, 


"This script will deploy, start, and test BookOrdersApplication (you will 
consider it version 1). The application will be running continuously, 
even while you are deploying version 2; therefore, old instances will 
not be affected since they will be running with version 1. 





4, Open the solution C: \APRESS\Chapter7\SideBySide\, 
BookOrdersSolution\BookOrdersSelution.<ln. 


You will simulate a new version (version 2) of fookOrdersSolution by 
performing the changes detailed in the following steps 
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Open the schema InconingBookOrders..xsd and add a new 
Element Field called Orderstatus. The idea here is that you 

are forcing a change in the incoming message so that original 
consumers can still send the old messages and new consumers 
will initiate the process by sending the new format of the message. 
See Figure 7-3. 








Figure 7-3. Examining the OrderStatus element 


6 


Change the minOccurs and MaxQccurs properties of Orderstatus 
to 1 By implementing this change, this field must be present in 
the incoming message; otherwise, message validation at pipeline 
level will fail. 








Open the schema CRMInconingBaokOrders.xsd and add new 
Element field called CRMOrderState. This action will wrap the 
OrderStatus into the simulated CRM application. See Figure 7-4. 





Figure 7-4. Examining the CRMOrderState element 


s 


Change the minOccurs and MaxOccurs properties to 1. Now the 


eld is mandatory. 





Open the map Map_InconingMessage_To_CRM_ 
Congratulationlessage.btnand create a new link button for the 
two new Element fields. See Figure 7-5. 
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Figure 7-5. Examining the OrderStatus map link 


10. Open the Map_IncomingBackOrders_To_CR¥Fornat.btm map 
and create a new link button for the two new Element fields. See 
Figure 7-6. 











Figure 7-6, Examining the Order status new map link 


1L. Open the orchestration and, just after the script shape 
SetPerfCounters, add a new script shape and name it iri teLog. 
Add the following code: 





Systen.Diagnostics.EventLog.triteEntry("Orchestration 
Version 2", "new Message Received in orchestration version 
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‘This action will add a new log entry in the Windows Application Event 
log so you can later see that the orchestration version 2 is processing 
messages. 

is time to change the assembly version. Go to Project properties, 
select the Application tab, and click on Assembly Information 

button. Choose version 2 forthe assembly and file versions and 

click OK. See Figure 7-7. 








Figure 7-7. Examining the assembly and file versions 


n. 
n. 


is. 


Choose Save All. 
Build the solution, 


Deploy the solution without stopping the old 
BookOrdersSolution 


Observing Both Versions 


‘To understand how BizTalk Server side-by-side execution works, let's examine the 
differences after deploying application version 2. 

Open the BizTalk Server Administration Console and refresh. As a result you still 
have only one BookOrdersApp! ication, However, if you explore all the application 


sections you will notice some differences, Go to the Resources section and observe the 
‘changes shown in Figure 7-8. 
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Figure 7-8, Examining the Resources section 


‘There isa new resource for the BockOrdersSolution assembly, and the 
version is 2.0.00, 
Explore now the Maps section also, as shown in Figure 7-8, 
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Figure 7-9. Examining the maps section 


ma 


CHAPTER? DECREASING DOWNTIME 


You can see now version 1.0.0.0 and 2.0.0.0 in the same application. 
Explore the schemas section, as shown in Figure 7-10. 
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Figure 7-10. Examining the Schemas section. 


‘The Property schema and the two business schemas have a new version deployed. 
Next, explore the Orchestrations section, as shown in Figure 7-11. 














Figure 7-11. Examining the Orchestrations section. 
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‘There isa new version of the orchestration. Notice that the version 1 is still up and 
‘running, That means that existing running instances will not be affected by the | 





anges, 
Now, open Performance Monitor and look for the new version of the counters. You 
‘have applied versioning also at performance counter level as shown in Figure 7- 
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Figure 7-12. Examining the versioned performance counters section 


Orchestration version 2 is creating versioned counters, as the 
PerfornanceCountershelper class supports that feature, 


Enabling Both Orchestrations to Run 





by Side 


As soon as you deploy the new version, like business rules, BizTalk Server will invoke the 
latest assembly version available, That means that once the application receives a new 

transaction, the BizTalk Server engine will parse the message as version 2.0.0.0, This will 
bea problem for the old orchestration, as it is linked to use version 1 of the assembly 
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A first glance, this behavior might look wrong, but it makes all the sense of the world, 
as the idea is that new instances will be redirected to the new version only. Ifyou need 
the old version to work with new instances, you must adjust the docunentSpechanes. 
property ofthe receiving pipeline. 

Follow these steps to change the decunentSpecNanes property. First, set the 
Docunent SpecNanes property of the receiving pipeline to the old schema version. Then, 
To generate a valid dacunentSpecNanes property, do the following: 


1. Using the BizTalk Administration console, go to the application 
schema section and locate the assembly containing version 1. See 
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Figure 7-13. Locating the assembly with the older version 


2, Right-click the schema and choose Properties, 


3. Copy the content ofname and assembly information into a 
notepad, as shown in Figure 7-14, 
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Figure 7-14. Obtaining the schema and assembly names 
4. Concatenate both values into one, separated by a coma, like this: 


BookOrdersSalution.schenas..IncomingBookOrders,BookOrders 
Solution, Version«1.0.0.0, Culturesneutral, Publickey 
Token=e82fb5e911d58f5b 


5. Using the BizTalkAdministration console, go to 
BookOrdersApplication, locate the receive location 
riBookOrdershpp, and access the properties. 

6. Gotothe receive pipeline section and click on the ellipsis button, 

7. Set the DocunentSpectanes property to the content you created 
using Notepad (see Figure 7-15): 
BookOxdersSolution.schenas. InconingBookOrders,BookDrders 
Solution, Version=1.0.0.0, Culture=neutral, PublicKeyTok 
enseB2Fose911d58f50 


EI 


CHAPTER? DECREASING DOWNTIME 
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Figure 7-15, Setting the DocumentSpecNames property 


d. ClickOK, 





Note Ifthe OK button is not enabled, change the cursor to any other property and 
the button will be enabled again. 





9. ‘Test the solution, now dropping a file for version 1. To do it, 
run this BAT Ble: C:\APRESS\Chapter7 \BookOrdersSolution\ 
BookOrdersSolution\Ports\SendiVIPBaokOrder. bat 


am 
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What Happened? 


The orchestration version 1 worked perfectly, but you get an error in version 
that it is expecting version 2: 





saying 


Received unexpected message type ‘BookOrdersSolution.schenas. 
IncomingBookOrders, BookOrdersSolution, Version=1.0.0.0, Culture=neutral, 
PubliickeyTokense82fbsegi1d58f5b' does not match expected type 
"BookOrdersSolution. schemas .IncomingBookOrders, BockOrdersSolution, 
Version-2.0.0.0, Culture=neutral, PublicKeyToken-eB2fbse11dsBfsb" 


You are now getting this error because, in the previous action, you told BizTalk Server 
10 parse the incoming message as version 1, So, version 1 of the orchestration can still be 
running and picking up messages. If you drop a version 2 message now, you will get the 
‘same error because both messages have the same message type property. 


How Do You Fix This Situation? 


Ifyou want to maintain the same targetnanespace property for both versions ofthe 





ma, the only solution is to create a new receive port with a different. 
2. With this approach, you will have both versions running in. 

parallel. Therefore, you will need to inform your source systems that the old version of 

the application will be available until a certain point of time only. After that, you will 

» will only work with versio 


eive location 





for orchestration versio 








disable the original receive location and the syst 





Using Business Rules to Reduce Deployment 


To reduce th 





number of deployments, you can always consider using the Business 
Rules Engine to separate certain business functionalities from orchestration processing 





"The Business Rules Engine allows policies to be changed in real time. That means 
thatar 





orchestrations that consum 





e those business rules do not need to be adjusted 
When the business policy changes. Developers will change business rules by creating 
new versions, and orchestrations, by default, will consume the latest version available 
without the need to redeploy the application again. 

Developers can create business rules using a tool called the Business Rules 
‘Composer (or using Business Rule APIs). Once published, the policy cannot be 


modified directly. To perform a change, a new version of the policy must be created. 
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Once deployed, the policy can be called from a BizTalk orchestration and all the policy 
evaluations will take place, 

Business rules are put together into a single policy definition. That means that a 
policy can has 
Conditions are evaluated, and if they return true, a collection of actions are considered 





e one or more rules. A rule is composed ofa condition and an action, 


Tor later execution. 





Note Keep in mind that in business rules there is no else option available. 
Therefore, if you want to also perform an action when the condition is not met, 
you have to add a new rule and create a new condition that encapsulates the else 
situation, 





Actions are not executed immediately after the condition is evaluated. Instead, the 


Business Rules Engine will add that action to a temporary agenda and, when all rules are 








evaluated, all actions within the agenda are executed based on priority 





Important Priority is designed to sort the actions by priority, not by the condition. 





"The source of information for a business rule is called a fact and developers define 
them into vocabulary definitions. Using the Business Rules Composer, you can create the 
following types of facts: 


‘+ Aconstant value, a range of values, ora set of values 
. A NET class orelass member 


. An XML document element or attribute 





. A database table or column. 


Policy Execution Steps 


"The Business Rules Engine follows these steps to execute a poli 





1. Allfacts re obtained. 


2. All rules are evaluated. 
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3. Actions of the rules that evaluate to true are added to the list of 
actions to execute. 


A, Actions are executed in descending order by priority. That means 
that the higher-priority actions are executed first. 


Facts are disposed. 





Note Ifan action changes a fact that is used as a condition of a rule, the rule will 
be evaluated again. 





Business Rules Performance Recommendations 


There are several recommendations and best practices that you should be aware ofto 


implement business rules successfully. 





Fact Types 


The Business Rules Engine uses more resources to access XML and database facts, 
‘Therefore, whenever possible, use NET facts to improve performance. 





Database Types 
To access a data from a database, developers usually create database facts within a 
vocabulary by adding a new database definition to 

You can fallow these steps o crete a fact that accesses database information: 


1. Using the Facts Explorer, select the Add New Definition option, as 
shown in Figure 7-16, 
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Figure 7-16. Adding a new vocabulary definition 
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2. The Vocabulary wizard will appear. Select Database Table or 
Column, as shown in Figure 7-17. 
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Figure 7-17. Adding database vocabulary definitions 
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3. The wizard will now show the data Binding Type options, as 
shown in Figure 7-18, 
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Figure 7-18, Setting the binding type 
4, To access a database to retrieve data, the Business Rules Engine 
provides the following database types: 


. DataConnection—Represents a table in a database accessed 
through a database connection, 


+ TypedDataRow—This type is based on the ADO.NET DataRow object, 
+ TypedDataTable— Represents a collection of TypedDataRow objects. 
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‘The TypedDataTable type loads all data into the memory of the server, which can 
‘cause a memory proble 

of TypedDataTable: 
large amount of data. For that propose, use the DataConnectionType, as it works more 
efficiently. 


Ifthe polled data is large. Therefore, you should avoid the use 








scenarios where the Business Rules Engine needs to retrieve a 


Caching Settings 


"The Business Rules Engine caches policies to improve performance, The first time à 
policy is requested, the policy objectis stored in the cache. BizTalk Server then loads 
that object into memory and executes it. The subsequent policy requests are using the 
cache object that is saved into the memory, reducing the execution time. The maximum. 
number of objects in the cache is controlled by a setting called CacheEntries and by 
default is set to 32. I the number of frequent policies is higher than 32, you can increase 
this setting to improve performance, 

By default, the Business Rules Service checks for changes to the rule every 60 
seconds, If there are changes, the service will update the cache object ta reflect the 
changes. The time of he checking cycle is regulated by the Pol LingInterval setting. 
your policies don't change at all or change every other month, you should consider 
increasing the PollingTnterval value. This will save CPU resources as the Business 
Rules Service will check for updates with less frequency. 

after one hour, a policy is not requested anymore, the Business Rules Service 
removes it from the cache. This can be tuned by changing the CacheTineOut setting. 

You can adjust these settings by adding the Microsoft RuleEngine section under the 
configSections section: 





«configuration 
«configsections» 
«section nanes"Hicrosoft.RuleEngine" type-"System.Configuration. 
SingleTagSectionHandler" /» 
</configSections> 
Microsoft RuleEngine 
Cachetntries-"120" 
CacheTineout="1860 
Pollingrnterval="30" 
h^ 
configuration. 
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Securing Application Configuration Settings 


Enterprise applications usually have to access configuration data that, in most cases, 
very sensitive. Eventually, you will need to store connection strings, user information, 
passwords, and settings that change the behavior of your applications like debugging 
‘modes, performance analysis, and things like that In today's world, protecting this 
information in a secure storage system decreases the chances of undesired attacks, 
BizTalk Server relies on the Enterprise single sign-on database to store all the sensitive 
data related to the engine configuration, such as adapters, ports, and application-related 
settings. This database is already encrypted with a very strong key, and the good news is 
that you can use it tø store applications settings. 

Figure 7-19 illustrates the Enterprise single sign-on database and shows how you can 
use it by developing a custom NET component that will access the store. 
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Figure 7-19. Understanding the SSO database to store settings 
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The SSO MMC Snap-In Tool 


Around 2008, the BizTalk Server product group team released an SSO Configuration 
Application MMC snap-In that can be used to create the configuration values within. 
the stare. You ean download it at https: //wwi.sicrasoft.con/en-us/download/ 
confixmation.aspx?ide14524. 

Once you have downloaded it, you will see the following files: 





^ SSOMCSnapInSetup.zip—Contains the installation files. 

+ SSOConfgurationipplicationMSBuildInportTask-—Ifyou want to 
integrate the tool with MSBuild, you could reuse the script and the 
DLL provided. 

+ $S0ConfigurationApplicationClientHelper—It contains a Cé class 
that will help you build your helper to access SSO database config 
settings 

Follow these steps: 

L Add your user to the SSO Administrators group. 

2. Unzip the SSONMCSnapInSetup zip fle. 

3. Run Setup.exe. 

4. Typeyourcompany name. This value will be used by the tool to 
create the default application 


5. Click Next and then Finish, 


Using the Assembly Binding Redirect Feature to Point 
to the Right Assembly 


Unfortunately, at the moment of writing this book, the tool has not been updated to 
‘work with the latest BizTalk Server versions, The Enterprise single sign-on feature uses 
the assembly called Microsoft. EnterpriseSingleSignOn. Interop dll, and for BizTalk 
2016, the current version is 10,0,1000.0, However, the SSO MMC Snap-in uses an old 
version 5.0.1.0, which is for BizTalk 2008, and if you attempt to run the tool without any 
modification, it will fail. 
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Luckily there is a NET Framework feature called Assembly Binding Redirect. By 
using this option, you can redirect an assembly to a different version, You can change 
the MMC snap-in config file to use the latest Microsoft. Enterprisesinglesignon. 
Interop.dll version. 
Follow these steps to doso: 
1. Using the File Explorer, go to C:\Program Files (x86)\ 
Microsoft Services\$S0 Application Configuration and edit 
the SSCMICShapIn. 1. config fle. 





Under the Configuration setting, add the setting shown in 
Figure 7-20. 


«configuration. 
‘cappSettings> 

‘<add keys"ConpanyNase" value="Developi 
</appsettings> 


BizTalkSolutions" /> 





— 
VassenblyBinding mlns-"urn:schemas-microsoft-com:asm. vi" 
appliesto="v4.0,30319"> 
“cdependentAssenbly> 
VassenblyIdentity nases"Microsoft. EnterpriseSinglesignon.Interop* 


publLickeyToken="31b¢3856ad364035" culturec"neutral" /> 
bindinghedirect oldVersion="5.9.1.0" newierslons"10.9.1090.0"/» 
</dependentAssenbly> 
<fassenblyBinding> 
«iruntine» 








configuration» 
Figure 7-20. Enabling the assembly redirect option 


3. Savethefile 
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Adding New Keys to the SSO Store 


"This section shows you how to add new keys to the SSO store. The key conceptis very 
similar to Ce hash tables, where elements are defined with a key and a value. 
Follow these steps: 
1. Open the SSO Application Configuration Tool, located by default here: 
C:\Program Files (x86)\Wicrosoft Services\$S0 Application 
Canfiguration\Ss0 Application Configuration.nsc. 


2, Right-click at the main root level, in our case that’s 
DevelopingBizTalk2016Solutions, and choose Add Application, 
as shown in Figure 7-21. 











Figure 7-21. Adding an SSO application 


3, The tool will set the default name to newApplication, Right-clickit 
and choose Rename. Set the name to BookOrdersAppl cation. 
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4. Right-click at BookOrdersApplicationlevel and select the Add 
Key Value Pair option, as shown in Figure 7-22. 





Figure 7-22. Adding a key value pair 


5, Set the key to ForceCountersCreation and the value to 0, as 
shown in Figure 7-23. Click OK. 





p x 
D 

















Figure 7-23. Setting the value for the key. 
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6. Createa new key called CongratulationMessage and set the value 
to "Congratulations, you are a VIP Customer? You will use this 
later in the VIP customer map. 


7. Aner the creation of these two keys, your sereen should look, 
something like Figure 7-24, 




















Figure 7-24. Examining the SSO application with the related values 


Importing and Exporting Application Configuration 


‘You can also import and export your configuration settings through different 
environments using a secure mechanism. 
Follow these steps to export the SSO application settings: 


1. Right-click the application you want to export. 
2, Click on the Export Application menu item. 


3, The Enter Key for Export dialog will appear. Type BookOrderskey 
as the encryption key, as shown in Figure 7-25. 
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Figure 7-25. Examining the SSO application with the related values 


This key is used to encrypt the configuration data that is written to 
the hard drive. You must use this same key when you import this 
application. 


4. Choose a location for the export file. Ifyou open it, you will see the 
encrypted data. 


Follow these steps to import the SSO application settings: 
1. Right-click at the root evel. 
Choose Import Application. 


3, Locate the SSO fle that contains the application settings you want 
to import. 


4. Type the key used when exporting the settings (see Figure 7-26) 
5. ClickOK, 
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Figure 7-26. Importing the encryption key 





Note If that application already exists, you will see new koy/pairs added to the 
existing application. 





Creating an SSO Helper Component 


The tool comes with a Cé class that you can reuse to create your own custom SSO helper 
component. In this section, you learn how to incorporate this component in your BizTalk 
Server solutions to access secure configuration data. 

Follow these steps: 


1. Open the BookOrders solution located at C:\APRESS\Chapter7\ 
0ConfigStare\BookOrdersSolution\BookOrdersSolution.sLn. 





2, Adda new class library Visual Ce project called SSOCLientHelper. 


3. Using File Explorer, locate the SSOCLientHelper class file that 
‘comes with the tool you downloaded previously. 





4. Copy the class implementation from the SSOCLientHelper class 
and replace itin the default class2.cs file in the new solution. 


5. Rename the classi.cs fil 





to $S0CLientielper. 
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Add the following references to the project: 

‘+ C:\Program Files\Common Files\Enterprise Single Sign- 
On\Microsoft.BizTalk. Interop.550Client.d11 

+ C:\Program Files\Common Files\Enterprise Single Sign- 
On\Microsoft EnterpriseSingleSignon.Interop.dll 

Add the following using statements to the SSOCLientHelper class file: 

using System; 

using System.Collections. Specialized; 

using Microsoft. BizTalk.$SOClient. Interop; 

Locate the SSOCLientHelper class definition, add the serializable 

atribute 

BizTalk Sever map: 





and 





Serializable] 
public class $50ClientHelper 
{ 


Sign the ass 





nhly using the project properties. 
Choose Save All. 


Bulld the project. 


Using the SSO Client Helper Component to Access SSO Data 


In this example, you are going change BookOrdersAppLication so the map will use the 
SSOCLientHelper component to access the BookOrders application SSO keys. This will fll 


the Observations Elem 





nl feld with the text specified by the CongratulationNessage key. 


Open the BookOrders solution located at C:\APRESS\Chapter7\ 
S0ConfigStore\Book0rdersSolution\BackOrdersSolution.sln, 


Add a reference to the SSOCLientHelper project created in the 
previous section. 
Open the orchestration called orcProcessBookOrders. 
Open the Map_IncomingMessage To CR. 
CongratulationMessage.btmmap. 
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5. Deleteallthe existing links and functoids related to the 
CRMObservations element feld. See Figure 7-27, 








Figure 7-27. Changing the map to retrieve the SSO configuration values 


6, Adda new scripting functold and set up and configure it as follows. 


Inthe Functoid Inputs tab, add two Input parameters (sce 
Figure 7-20) 


a Input[o]:Value BookOrdersApplication 
bs Input[a]: Value Congratulationtessage 
In the Script Functoid Configuration tab set the following values: 
a. Script Assembly: SSOCLientHelper, versionei.0.0.0 
b. ScriptClass:SSOCLientHelper.SSOCLientHelper 
c SeriptMethod: Read 

7. Choose Save AlL 

4, Choose Build A 


9. Run the command file C:\APRESS\Chapter7\SSOConfigStore\ 
BookOrdersSolution\Setup\SetUp.crd to deploy and test the 
application. Ifyou did not go through all the previous steps, this 
CMD fle will just deploy the book orders base solution, which 
does not contain SSO functionality. Additionally, Ir your solution, 
does not compile, this CMD file will also fail. 
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Figure 7-28. Calling the SSO helper component in a map 


Summary 


‘Decreasing downtime is a major issue when developing mission-critical solutions that 
require processing messages 24x7, 265 days a year In this chapter, you learned how to 
decrease application downtime using side-by-side versioning and business rules to 
derive the business Now from orchestration to the rule's engine, 

Even though this book does not focus on security, this topic is a recurrent 
requirement for the majority of BizTalk Server applications that rank higher on the 
application priority levels, thus I decided to include the topic about storing configuration 
data in the SSO database. 

In the next chapter, you learn how to test your BizTalk Server solutions. 
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Monitoring Using 
BizTalk 360 


"This chapter is focused on using the BizTalk 360 tool to monitor the BizTalk Server 
platform. Once your BizTalk solutions are deployed into the production environment, it 
is important that the BizTalk Server team be aware ofthe issues that are occurring on a 





daily basis to proacth you need to be 





increase the health of the platform. For instanci 





notified ifa receive location or host is down. 

The BizTalk experts at BizTalk 360 have i 
faced in the past 10+ years. BizTalk Server customers usually build custom management 
solutions on top of BizTalk to address all these challenges. Well, in this case, you do not. 
have to reinvent the wheel, as BizTalk 360 extends BizTalk Server monitoring to a point 
where most scenarios are efficiently covered. 





ntiied challenges that customers ha 





Understanding Monitoring in BizTalk 360 


In this section, you learn which types of monitoring exist in BizTalk 360 and how i 
can be setup. 








Different Types of Monitoring 


There ate several differ 





nt ways to monitor using BizTalk 360 and they are discussed in. 
this section, These possibilities are: 


‘+ Threshold monitoring—Used when a condition is violated, for 
‘example, a receive location that's expected to be enabled is disabled 
due toa temporary failure. 
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+ Health Check monitoring—Used, for example, to perform a daily 
checkat 9AM every business day. 


+ Data monitoring—Used, for example, to set up monitoring whether 
specific workloads are really being processed, BAM milestones are 
achieved, or alarms are raised based on suspended messages. 


Threshold Monitoring 


When a threshold parameter is exceeded, BizTalk 360 can be configured to redirect 
Immediate alerts. Ifyou want to monitor only during business hours, you can tell BizTalk 
360 to send alerts only duringa specifie time of the day. When the situation comes back 
to normal, you can set up the alert so that you will receive notification stating that the 
original condition has been solved. If the issue happens frequently, it might overload 
your inbox, so BizTalk 360 pro 
that will be sent. 





lesa mechanism to limit the number of notifications 


Health Check Monitoring 


Iis common practice in supporting a BizTalk environment for support people ta 
maintain daily/weekly routines to check the health of the environment, You can use 
BizTalk 360 to automate the daily health check process by creating a health check/status 
alarm that checks at specific times (for example, Mon-Fri, TAM). 


Data Monitoring 


BizTalk 360 provides you with query builders that will help you access special BizTalk 
Server information, such as the message box database to look for suspended service 
instances, latency information, and active instances. If your solution is implementing the 
business activi 





jonitoring feature, you can ereate queries to monitor BAM activities to 
adition is met. 





create specifie alerts when a certain 
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Setting Up an Alarm 


‘To configure an alarm, you n 





to follow these steps 


Basic settings: Provide information such as name, email recipients, 
and template email to use, 


Set the alarm type: The following options are available: 
^ Threshold, 

^o Health 

+ Data Monitoring 


Alarm mappings: Define the elements to monitor in this alarm. 


Configuring the Basic Settings 


Follow these steps to configure the basie settings for an alarm. 


2 


Login to the BizTalk 360 application asa Super Us 





Access the Navigation panel on the left side of the screen and click 
the Monitoring tab. 


Click the Manage Alarms tab. 


In the toolbar, click New Alarm and select the alarm type from the 
drop-down. 


^ Threshold 
+ Health 
* Data Monitoring 


‘The Alarm - Basic section appears, as shown in Figure 8-1. 
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Aarm- Basic x 
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Figure 8-1. Basic alarm settings 


am 


Fillin these fields: 
‘+ Alarm Name (required): A descriptive name of the alarm. 
^ Description: A detailed description on the purpose ofthis alarm. 


. Disable Alarm for Maintenance: This allows you to temporarily 
disable the alarm. As ong as the alarm is disabled, you won't 
receive notifications 
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+ Email IDs (required) The email addresses of the people who 
need to receive the email notifications. 
+ Email Template: Select one ofthe available templates. 
+ Enable Email High Priority: This option allows you to enable/ 
disable high-priority emails. Based on this configuration, the 


email notification priority is set. By default, high priority is 
enabled. 





+ Configure Email Template: By clicking on this button, you will be 
redirected to a screen where you can create new templates, make 


changes to a template, or delete existing email templates. 





+ Custom Notification Channels: Besides receiving notifications by 
email, you can receive notifications by other channels. Configure 
them here to receive notifications from this alarm. 





The Next button navigates you to the next screen. Click Next twice. 
until you reach the Alarm — Advanced pane (see Figure 8-2). The 
Alert - Threshold and Alert - Health panes are discussed in later 
sections, 








Figure 8-2, Examining the advanced options for an alarm 
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The Alarm - Advanced pane has the following fields: 


+ Send Notification SMS—Enter a mobile number (without 00 
ar the + sign to receive SMS Notifications). Note: BizTalk 360 
uses the Azure Gateway for sending SMS messages. Ifyou want 
to receive more than five (free) messages per day, you need a 
subscription to that gateway 


. Enable HP Operations Manager Integration- Select this field 
ifyou have an HP Operations Manager and want to receive 
notifications there. Note: HP Operations Manager Integration has 
to be configured under Settings/Monitoring and Notifieation/HP 
Operations Manager. 

+ Log Events to Event Viewer Select this if you want to write 
events from this alarm to the Event Log. Ifthe Event Log option. 
is enabled, down and up alert notifications are written to th 
[BizTalk 360 Monitor source in the application lg. 





^ Eventld—Onee the previous field is selected, this field is enabled 
and a numeric value is required. When a notification from the 
alarm is sent, an event og entry is written in the Application og, 
with the Eventld configured here. 


“+ Enable Test Mode- Select this field to run this alarm once per 
minute. If applicable, notifications will be sent. 


^ Previous Navigates back to the previous screen. 
. OK Stores the alarm in the BizTalk 360 database, 
6. Enter/select any of the optional fields and click OK to save the 
alarm. 
Threshold Alert 
Follow these steps to set up a threshold alert: 
L Locate an existing alert and click on the Edit button. 
2, Enable the Alert on Threshold Violation option. Otherwise, the 


threshold options are not enabled, 
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3. Use the Violation Persist Duration setting to tell BizTalk 360 that 
you want to receive notifications only when the issue has been. 
happening fora period of ime, For instance, you can receive 
notifications if the violation persists for 10 minutes, 

4. The Limit Alerts per Violation parameter limits the alerts sent 
about a specifie violation. For instance, Ifyou set it to 3, BizTalk 
360 will send a maximum of three notifications per violation 
persist duration period. 





"The Notification When Situation Becomes Normal setting tells you 
when the situation becomes normal after a violation is over. 

6. Setalerts on set day(s) and time(s) only to restrict the threshold 
violation alerts 





7. Save the alarm information. See Figure: 


‘arm -Threshold 


O mamaaa 





- E 
Prenat rat 
pres ae 





Figure i3. Examining the threshold alarm options 
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Health Monitoring Alert 
‘To set up a Health Monitoring alert, follow these steps: 
L. Follow the steps as mentioned in the Basic Settings section. 


2, On the Alarm - Health page, you have option to choose the days 
and times you want to receive periodic update about the health of 
your environmer 





3, You can click OK or Next to move to the last sereen to add 
some Advanced Settings like SMS configuration and advanced 
configuration, 


‘Once you finish, the alarm will be set, and the BizTalk 360 user interface will take you 
to the Manage Alarms page, This page lists all dhe alarms that y 
environment, See Figure 8-4. 





ju have created in your 





Alarm - Health 


B Health Monitoring Alert 





Figure 8-4, Examining the health monitoring alarm options 
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Data Monitoring Alert 
Follow these steps to create an alert for Data Monitoring: 

1. Follow the steps mentioned in the Basie Settings section, 


2. On the Alarm - Data Monitoring section, enable the Use This. 
Alarm for Data Monitor Alerts checkbox. 


3. Select the Notify on Success As Well checkbox if you want BizTalk 
36010 send a notification when the query runs successfully 


What Do You Monitor? 


As you are now aware what kinds of monitoring exist, you can start thinking about what 
Kind of artifacts you want to monitor. Ata high level, this can be separated into a few areas: 





^o BizTalk Platform monitoring 
. BizTalk Application monitoring 
+ Data monitoring 

+ Endpoint monitoring 


For each of these categories, BizTalk 360 contains multiple capabilities, most of 
‘which are explained in the following sections. 


BizTalk Platform Monitoring 


Your BizTalk solutions run on the BizTalk Server platform. So, when something is wrong 
with the BizTalk platform, that might affect how your BizTalk solutions run and, asa 
result, damage your business processes, Therefore, it is important to constantly have a 
good understanding of the health of your BizTalk Server platform. 

When you want to monitor your BizTalk environment, there are several things you 
should be aware of, and you ean use monitoring to watch these things, Consider these 
examples; 

+ BizTalkhost instances 


‘+ BizTalk Server's SQL Server jobs 
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+ Availability of the BizTalk Servers 
+ Certain Windows NT services 
^o Hostibrotling 


BizTalk 360 can help you monitor all these different artifacts and of course, when 





something is wrong with any of these components, you can become notified. In many 
‘cases, BizTalk 360 will try to automatically bring the components back to the expected 


Monitoring Host Instances 


As you learned in the previous chapters, host instances are the Windows services that 
‘encapsulate most ofthe BizTalk Server functionality, Providing a proactive monitoring 
mechanism to ensure that all relevant host instances are started becomes crucial to the 
health of the platform. BizTalk 360 enables you to monitor host instances by creating 
monitoring rules that check whether the service is started, stopped, or disabled. 

Additionally, ifyou have clustered host instances, BizTalk 360 can check ifthe host 
instance is active in more than one server 





Setting Up Monitoring for Host Instances 

To set up monitoring for host instances, follow these steps: 
1. Login to the BizTalk 360 application. 
2. Click Monitoring in the navigation panel 


3. Click the expand button of the Manage Mapping tab and select the 
BizTalk Environment link. 


4. Select an Alarm name from the drop-down for which you would 
like to associate the alerts. 


5. Select the Host Instances tab. 


6. Select the host instances you want to monitor, 





ju want ta reeelve 
1s (because 
should be always stopped, ike a FTP host instance), y 


7. Set the value of Expected State. For instance, ify 
a notification when a specific host instance 








u need to 
set the value to Started, See Figure 8-5. 
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Figure 8-5. Creating a host instance alarm 





Note From the Platform Health alarm, the BizTalkServerApplication host instance 
needs to be monitored to be in the Started state. In case the host instance is 
stopped, BizTalk 360 should automatically try to start the host instance, so Auto 
Correct needs to be Enabled. 





‘The Auto Correct Feature 


‘This is a very useful featur 
‘current state differs rom the expected state 
For instance, imagine that you have a host instance monitor that checks whether the 


as it will allow you to set a specifi state when the artifact 





status ofthe host instance is started or stopped. Ifyou created the monitor to expecta 
Started state, and if you enable the auto-correct functionality, BizTalk 360 will attempt to 
start the host instance if the current state is Stopped. 

rect for a host instance, just follow these steps: 





To enable auto-c 





1. Select the desired host instances, 
2. Select Started in the Auto Correct drop-down. 


3, Saveyour changes. 
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Monitoring Host Instances That Are Clustered 


‘To provide high availability for FTP adapters, for instance, Microsoft recommends 
clustering them. BizTalk 360 has the capability to mor 





Justered host instances, 
"The difference with non-cluster host instances from BizTalk 360's point of view is that a 
clustered host instance should be marked with a state of AtLeastOneActive. BizTalk 360 
Will check if the host instance is started in only one instance. See Figure 8-6, 





—* 

















Figure 8-6. Monitoring clustered host instances 


Monitoring SQL Server Jobs 


As you have learned in the book, BizTalk Server uses SQL Server jobs to make sure tables 
within the Message Box and Tracking DTA databases are consistent If any of these 
jobs fail, the engine will start behaving unexpectedly and most likely performance will 
decline, as internal tables will grow aver the time. 

Luckily, BizTalk 360 enables you monitor all SQL jobs by paying attention to two 
important threshold 





+ The Job Sta 





Which is used to check ifa job is running or not, 


+ The Last Run State, which is used to diagnose the latest 
execution state 
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Al BizTalk Server jobs must run successfully but these: 


+ MessageBox Message Cleanup BizTalkMsgBoxDb—This job should 
be disabled because is being called from within the MessageBox _ 
Message ManageRefCountLog_BizTalksgBoxDb logic. 


+ MessageBox Message ManageRefCountlog BizTalkspBoxDb—Can. 
show an execution error if the job or SQL Server Agent has been. 
manually or unexpectedly stopped. The job calls a stored procedure 
that runs in an infinite loop. This behavior is by design and cannot 
be changed. Because of this, DBAs usually disable this job so it does 
not raise any errors, which is a practice not supported in the BizTalk 

product support information, 





‘Therefore, keep this in mind when creating monit 
accordingly: 





ig for these jobs and act 


+The Expected Job state for the MessageBox Message Cleanup - 
BizTalkMsgBoxDb job should be disabled. 


‘+The Last Run state for the job MessageBox Message - 
ManageRefCountLog_BizTalkMsgBoxDb should be error. 
Establishing Monitoring for SQL Jobs 
Follow these steps 
L Select SQL Server Instances in the Manage Mapping section. 


2. Select the SQL Server instance that runs the SQL Server job you 
want to monitor, 


3. Select an educated alarm. 


4. Select the desired SQL jobs. 


Set the value of the Expected Job state and the Expected 
Last Run state 
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Monitoring the Availability of the BizTalk Servers 


Ina real-world scenario large enterprises will typically have or 
deployed BizTalk environments. To handle such scenarios with BizTalk 360, you can set 
up monitoring on specific components of BizTalk Server such as BizTalk applications. 
and their associated Send Ports, Receive Locations, Orchestrations, Service Instances, 
Disks, Event Logs, NT Services, System Resources (CPU, Memory), SQL Jobs, Web 
Endpoints, BizTalk Health Monitor Errors, and Warnings and Host instances (normal, 
clustered) See Figure 8-7. In addition to these existing monitoring features, BizTalk 360 
has the capability to monitor the BizTalk Server availablity 


e or more sets of 

















Figure 8-7. Monitoring the servers of the group 


Why ity Monitoring in BizTalk 3602 


Say that ACME Corp has a highly complex BizTalk Server group with five BizTalk Servers 
running 24/7 operations. Typically these complex BizTalk Server groups are configured 
Tor high availability and scalability, keeping in mind the high volume of tafe. With such 
setup, administrators will be under a lot of pressure to make sure all the BizTalk Servers 
are up and running and can process all the messages atthe expected level, In any 
complex BizTalk Server group configuration, there is a high likelihood that, fo 





"Talk Server Avai 
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seasons, one or more servers will go dow 





Tt becomes a daunting task for the BizTalk 
administrators to continuously monitor the server availability and react to downtime 
quickly. To help administrators overcome this challenge, BizTalk 360 provides the 
BizTalk Server Availability Monitoring functionality. See Figure 8-8. 











Figure 8-8. BizTalk Server Availability Monitoring 


Important Points to Remember in BizTalk Server Availability Monitoring 


While setting up BizTalk Server Availability Monitoring, there are a few things to be 
aware of 

+ Choose the ICMP or Telnet protocol. To check the BizTalk Server 
availability, administrators need to reach (ping) the servers. BizTalk 
360 supports Ping and Telnet; therefore, one of these protocols needs 
to be enabled on the BizTalk Servers. 

+ BizTalk administrators can choose when to receive the alert—when 
tone of the BizTalk servers in the group has gone down or only when 
all the servers in the group have gone down. 

* In BizTalk 360, BizTalk Server availability can be configured for only 
one alarm per environment. 
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+ When a gateway proxy ls configured in BizTalk 360 (under BizTalk 360 
Settings), Telnet uses the proxy detalls to bypass the firewall. 


+ BizTalk 360 also gives you the option to modify the existing 
configured server availability monitoring at any time, 


Configuring BizTalk Server Availability Monitoring in BizTalk 360 
Follow these steps to set up BizTalk Server Availability Monitoring: 
1L. Login to BizTalk 360. 
2. Clickthe Monitoring section from the top of the left Navigation menu, 
3. Click the + button (the Expand button) next to Manage Mapping 
and select the BizTalk Servers option, 
4. Click on Configure BizTalk Server Availability Monitoring on the 
top right. By doing this, it will automaticaly list all the configured 
BizTalk Servers. 
5. Select the alarm name from the drop-down that you would like to 
associate with the BizTalk Server for monitoring. 
6. When there is no BizTalk Server configured for monitoring, the 
value of Monitor Status will be be to Not Configured. 
7. By default, the ICMP protocol and Any One Server Are Down. 
‘options will be selected. 
3. Select the BizTalk Server that you want to monitor and click 
Enable Monitoring to start monitoring the BizTalk Server. 
When the settings are configured, BizTalk 360 will start monitoring the BizTalk 


Servers for any state-based violations. Once a violation occurs for the configured alarm, 
BizTalk 360 will notify the users via the configured notification channels, 


Monitoring Windows NT Services 





‘As you have seen throughout the book, BizTalk Server uses these Windows: 
+ The Enterprise single sign-on service 
+ The BAM Alerts service, if BAM alerts are enabled 
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+The Rule Engine Update servi 
implemented 


s i business rules are installed and 


^o Internet information server service to expose and consume services 





You can monitor those NT services for the following two different states: 





^ Running 
+ Stopped 
Follow these steps to set up monitoring for your Windows NT services 


1. Login to the BizTalk 360 application. 





Click Monitoring in the navigation panel. 


3. Click the expand button next to the Manage Mapping tab and, 
depending on your requirement, select the BizTalk Servers or the 
SQL Servers ink 





4. Select the BizTalk or SQL Server for which you want to set up 
monitoring on NT services, 


5. Select the alarm name you want to use. 
6. Select NT Services tab at the top of the page. 


7. Select the: 





vant NT services 
3. Select the checkbox on the NT services to activate the alert 


3. Set the expected state (started or stopped) by selecting the value 
from the drop-down. 


Viewing and Monitoring Host Throttling 


"The idea behind the BizTalk 360 Throttling Analyzer is to simplify the complexity in 
understanding the BizTalk Throttling mechanism and provide simple dashboard 
View. The purpose of host throttling monitoring is to notify the customers if there is any 
throttling happening in their environment. 
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‘The BizTalk 360 Throttling Analyzer 


"The BizTalk 360 helps solve the following problems: 


‘+ Asyou learned in Chapters 2 and 4, throttling situations can. 
become complicated to analyze even for seasoned BizTalk Server 
professionals. 


+ BizTalk Server only provides performance counter data, You have to 
use external tools, such as performance monitor or PAL, to analyze 
the raw information. 
"The objective of BizTalk 360 is to provide a visual representation of what is occurring 
In the envionment ina near real-time situation. 


‘The Throttling Performance Counters Collection Service 





BizTalk 360 provides a monitoring service that collects throttling performance counter 
data and stores it into the BizTalk 360 database. The default collection interval is 15 
seconds, The data is persisted for seven days, and users cannot modify it 

The throttling analyzer is in the Analytics tab. The service status of the Throtling 





Analyzer can be accessed from Settings > Analytics Health > Analytics Service Status. 


Setting Up Monitoring for Host Throtting 
Follow these instructions to set up monitoring for host throttling (see Figure &-9): 
1. Login to the BizTalk 360 application 
2, Click Monitoring in the navigation panel 
3. Expand the Manage Mapping tab and select BizTalk Environment. 
4. Select an alarm name. 


5. Select the relevant hosts. For instance, if you want 
to monitor BizTalkServerApplication, select the 
BizTalkServerApplication checkbox and click Enable 
Monitoring. 


6, BizTalk 360 will automatically start monitoring the host and 
looking for throttling events. 
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7. Click the Edit button and configure the threshold alert for 

publishing and delivery throttling. 
3. Click the Save Configuration button, 





Figure 8-9, Setting up monitoring for host throttling 





Note From the Platform Health alarm, the BizTalkServerApplication Host needs to. 
be monitored for throttling. In case of throtling due to publishing, a warning should 
be sent if the duration of the throttling condition is longer than 60 seconds. 








BizTalk Application Monitoring 


A BizTalk environment usually contains several BizTalk applications that use most of the 
BizTalk Server artifacts and elements, such as receive locations, orchestrations, ports, 
and business rules. At any given time, an issue related to these artifacts can occur, which 
leads to business process interruptions. 

‘To prevent you from monitoring all these elements reaetively and manually, BizTalk 
360 allows you to monitor all these artifacts in a proactive way. The following sections 
guide you through the process of enabling monitoring for all these BizTalk Server 
elements. 
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Receive Locations, Orchestrations, and Send Ports 
In BizTalk 360, you can monitor these BizTalk Server artifacts by checking whether the 
statusis disabled or enabled, You also can choose to not monitor the artifact at al 
Configuring Alerts for BizTalk Artifacts 
Follow these steps to monitor BizTalk Server artifacts: 

l. Click on the Monitoring panel 

2, Expand the Manage Mapping tab and select the Applications link 

3, Select the application that owns the artifact, 


A. Selectan alarm name. 





5. Select all relevant artifacts, such aso 
locations, or send ports. 


6. Set the value ofthe Expected State 





7. Enable the Auto Correct functionality for the receive location if 
you want BizTalk 360 to change the status of the receive location. 


when they are in a state that's different than the expected one. 


Figure 8-10 shows how the interface looks when setting receive locations 








Figure 8-10. Monitoring receive locations view 
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Figure 8-11 shows the Orchestrations alarm view. 











Figure 8-11. Monitoring orchestrations view 


Figure 8-12 shows the Send Ports Alarm vie 
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Figure 8-12. Monitoring send ports view 


Service Instances 


Fora healthy BizTalk environment, itis important to keep an eye on the number of 
instances and messages are stored in the Message Box database with a specifie state that 
‘can vary along the life of the instance. 


e Instances in the environment. As you have seen in previous chapters, service 
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based on the in 


‘The BizTalk administration console will display the following service states 
+ Ready to Run 
+ Scheduled 





+ Suspended, Resumable 
+ Suspended, Non-resumable 
+ Active 

+ In Breakpoint 


Using BizTalk 360, you can monitor those states and set up different threshold levels 





Setting Up Alerts for Service Instances 


Folk 
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low these steps to set up monitoring of service instances: 
1. Click Monitoring in the navigation panel. 
2, Expand the Manage Mapping tab and select the Applications link. 


3. Select the application that work with the service instances you 
want to monitor, 


4. Select an alarm name, 





5, Select the service instance states that you want to receive 


notifications 


6. You can edit the warning and the error thresholds level by clicking 
the Edit link, as shown in Figure 8-13. 


os 
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Figure 8-13. Monitoring services instances view 


What Is Data Monitoring? 


The best way to understand this feature is to go through an example. Imagine that every 








day, early in the morning, you expecta batch of atleast 100 messages from a specific 


location. Ifthe messages do not arrive, business operations are adversely affected. If 

you have tracking enabled for that receive location, BizTalk 360 can access the tracking 

database to check whether the messages have arrived to your BizTalk Server or not 
BizTalk 360 can access data from the following integration layer elements: 


Tracking Data 
Message Box 
Business Activity 


Monitoring 


Electronie Data Interchange (out of the scope of the book) 


Enterprise Servic 


se Bus (out of the scope of the book) 


Logic Apps (out of the scape of the book) 


Event Log 
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Setting Up a Tracking Data Monitor 


Follow thes 


am 





steps to create à tracking-data monitoring alarm: 


Go to Data Monitoring in the left pane and navigate to tracking 
data. 


Select the Add New option, 
Choose the alarm you want to use, 


Usea meaningful monitor name. In the scenario described at the 
beginning ofthe section, you could use something like: Check if 
the 100 Message batch arrived. 


Select Tracked Service Instances for Query Type. 


You can filter the tracked service instances to get a specifie 
instance as you can do using the BizTalk Server administration 
console. The following options are available: 


+ Assembly name 
^o Assembly version 

^ Error code 

^o Error description 

^ Hostname 

^o Service class 

^ Service instance ID 
^ Service name 

+ State 


The Warning Threshold setting configures when BizTalk 380 
should send the notification as a warning. For instance, ifthe 
number of tracked service instances is greater than 100, throw a 
warning 

“The Error Threshold setting is similar to the warning section, but it 
sends an Error notification, 


10. 


m 


n. 
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In terms of frequency, decide if you want to monitor the host 
instances daily, weekly, or monthly. 
Select the time to execute the alarm. 


Set an specific time of the day (detailed frequency option) or the 
end of the business day (end of business day option). 





Review all the advanced scheduling options available in this tool. 

"This book does not cover the advanced section because iis very 

specific and designed for more complex scenarios. 

Decide how do you want BizTalk 360 to validate the output data. 

‘You can base the analysis using the following options: 

^ Query result count (no date/time filter): The query will return a 
record count that you ean use to set up the thresholds. 

^ Query result with date/time range: You can filter the output data 
to checkif the records are generated on a specific time frame. 

Review the summary information justin case you missed. 

something. 


Click Save and Close. 


Message Box 


‘As you learned in Chapters 1 and 4, the number of suspended messages can impact 


negatively the performance of your BizTalk Server group. You can use BizTalk Server 360 
to automatically archive and/or terminate suspended instances. 


Archive and Terminate Suspended Service Instances 


you want to archive suspended. 





ssages for further investigation or just as a backup 


before automatic termination occurs, you have to configure the Archiving folder. 


Follow th 


1 
2 





steps to set up the Archiving folder 
Navigate to BizTalk 360 Settings 





Choose the System Settings option from the left mi 
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3. Edit the Archive Location for data monitoring setting to specify 
the Archiving folder. 
4. Click Save. 


You can now create a Message Box Data Monitor to terminate the 
suspended messages. Follow these steps: 


5. Goto Data Monitoring in the left pane and navigate to the 
Message Box. 


6. Select the Add New option. 
7. In the Ser Actions section, select the Is Action Required checkbox 
3. Depending on the requirement, select the When To Action option, 
3. Inthe What Action option, choose Terminate. 


10. Click Save and Close. 


Business Activity Monitoring (BAM) 


"The BAM portal in BizTalk 260 allows business users to query BAM views and perform 
activity searches. The concept of this feature is similar to BAM alerts, as you ca 
advanced filters based on the content of the BAM activity. This is extremely helpful if 
you want to receive alerts based on certain criteria when the Activity Completed table is 
populated within the BAM primary import database, For instance, you can create a data 
monitor alert when a book order has been denied or when an error occurred during the 
‘whole business flow (of course, only if the BAM activity is tracking errors) 


Event Log 


BizTalk 360 can monitor the event logs of your BizTalk and SQL Server boxes. This 
feature is useful not only to detect undesired BizTalk errors, but also to monitor custom. 
‘event logs raised by applications 





create 
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Endpoint Monitoring 


Besides monitoring BizTalk artifacts, se 
‘can also monitor resources like web services and FTP sites, which are part of your 
integration. BizTalk 360 allows you to monitor the following endpoints 


er resources, Azure services, and so ot 








«o Folder locations 
^ FIP,E1PS, and SFTP sites 
^o HTTP web endpoints 

+ MSMO 

+ IBMMQ 


. Azure servie 





Refer to the BizTalk 360 documentation to get more information about these topics, 


How Are You Notified? 


When a threshold is exceeded, the tool will rise a notification, BizTalk 360 exposes 


several functionalities to notify you when these situations arise. Ler's look at the different 





options 
+ Dashboards 
+ Operations Dashboard 
+ Monitoring Dashboard 
+ Dara Monitoring Dashboard 
^o Notification channels 
+ Email 


+ Other Notification channel 





+ Custom Notification channels 
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Dashboards 


BizTalk 360 has dashboards that show an overview of the platform's health, These 
dashboards are: 


+ Operations Dashboard 
* Monitoring Dashboard. 
* Data Monitoring Dashboard. 


Operations Dashboard 


"The Operations Dashboard provides access to key functionalities of the product that you 
‘will be using more frequently (see Figure 2-14). When the Dashboard loads, it shows a 
set of default widgets that will give you important insights on the environment. 





Figure 8-14. The Operations Dashboard view 
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Dashboards are customizable and you can add your custom widgets to redirect how 

users see the information. 

Monitoring Dashboard 


"The Monitoring Dashboard is used to populate all the alarms of the platform. The 
information displayed on this dashboard can be abundant, so it is recommended to have 
a big monitor to deal with the alerts efficiently. To access the monitoring dashboard, 





ollow the steps below: 


1. Clickthe Monitoring tab in the Navigation panel. 





2. Select the relevant alarm, 


3. Examine the alerts. 


Graphical Hierarchy View Structure. 


"The Monitoring Dashboard provides a tree view representation ofall the configured 
sections within an alarm and implements the following functionalities (see Figure 8-15): 


+ Clicking on element of the tee representation takes you to the. 
associated element. 


+ Youcanclickon the Toggle Fit to Screen option to adjust the 
visualization to your screen size Ifyou have many alerts set up, it 
might be handy to display the information ona large monitor. 


+The tool draws the information using these colored patterns: 
+ Red: The monitor returned an error. 
+ Amber: The monitor returned a warning, 


+ Green: The monitor did not find any error or warning. 
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Figure 8-15. The graphical tree view 


Data Monitoring Dashboard 


The Data Monitoring Dashboard populates the monitoring elements using a calendar 
view that helps you to understand which data monitoring took place and wher 
shown in Figure 8-16. 
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Figure 8-16. The monitoring dashboard view 


The tool will draw the information using the following color patter 





. Red: A data monitor returned an error. 
^ Amber: A data monitor returned a warning, 
. 6 





'n: All data monitors ran successfully during that period 


Viewing Execution Details. 

you want to extend information on a particular d 
ofthe cal 

Figure 





(ou can ellekon any area 





ndar and you will see what happ 





sd with much more detail, as show 
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Message Box Data Execution Results 
Tuesday, March 03,2015 
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ontiguted Query: Suspended Service Instances 
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Figure8-17. Execution detail view 


Automated Recovery 


BizTalk 360 has quite an extended set of artifacts that can be monitored and that can, 
send notifications. Besides that, it also has a number of automated recovery features. 
These features are: 


‘+ Auto-correct state bound artifacts: For example, bringing a Host 
Instance, which did not start after a reboot, back to the Started state 


*— Auto-resume: For example, resuming service instances that were 
suspended due to a network glitch. 


^ Auto-terminate: For example, terminating routing failure reports. 
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Summary 


In this chapter, you learned how to use the BizTalk Server 360 tool to extend the out-of- 
the-box monitoring features of BizTalk Server, not only from the availability and 
points of view, but also by creating custom aler 





ath 
that will help you monitor tracking, live, 





‘or even external data sources. 
In the next chapter, you read a BizTalk Server tale about a company that starts with 
BizTalk Server and evolves to a mature architecture by facing and solving problems. 
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CHAPTER 9 


Testing BizTalk Server 
Solutions 





BizTalk Server solutions frequently integrate highly critical business processes wl 
the integrated parties define very restricted service level agreements. Most of these 
applications rank high on the application priority level and must run 24x7 with 
downtime. In this situation, developing a successful esting architecture will make 

a huge difference in finding application bugs, performance bottlenecks, issues with 
new application versions, and in detecting problems when scaling the BizTalk Server 








platform. 

In this chapter, you learn the performance methodology required to assess a BizTalk 
Server environment and how to include unit and performance testing as part of your 
application development cycle. 


Unit Testing 


Unit testing is used to develop and execute testing applications to verify that your code 
is sunning as expected. I is called unit testing because the idea Is that you break down 
the functionality of the BizTalk Server solution into discrete testable pieces of code that 
you can testas separate units. In BizTalk Server, those units are usually identified by an 
individual message flow. 

Ifyou include unit testing as part of your development strategy, the quality of 
the application can increase by reducing the chances of bugs. Once you finish the 
development of a business flow, you should create unit tests to verify that the behavior 
‘of the application, with all the possible messages involved, is expected in terms of the 
business functional 





Furthermore, unit tests can be used later for performance testing. 
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When developing NET custom components, you can quickly produce test projects 
and test methods from your code, or you manually create the tests as required. 
You can leverage the following technologies to perform BizTalk unit testing 





‘+ Using the Unit Testing feature At the moment of writing this book, 
only the following BizTalk artifacts are available for unit testing: 


+ Schemas 
+ Pipelines 
+ Maps 


Unfortunately, orchestrations cannot be unit tested using the Unit Testing feature 
because they require the BizTalk Server engine to be executed. However, vou could test 
the end-to-end process by creating a general Visual Studio unit testing project, a tople 
‘that will be covered in the section “Assessing the Performance of a BizTalk Application 
later in this chapter. 








+ General Visual Studio Unit Testing Project—Ityou have Visual 
Studio Unit and Load Testing experience, this is probably the most 
convenient way to automate BizTalk Server Solutions testing. 





. BizUnit—BizTalk BizUnit it is a declarative XML framework designed 
to create unit tests for BizTalk Server projects. BizUnit Test Cases have 
three phases: Stage, Execution, and CleanUp. Every test is defined in 
a typed XML document that the BizUnit Engine will execute. BizUnit. 
has been used frequently over the past few years and it will not be 
covered in this book. You can review BizUnit documentation in th 
MSDN Microsoft page here: https: //docs.nicrosoft.con/en-us/. 
biztalk/technical-guides/using-bizunit-to-facilitate- 
autonated- testing. 


Creating Unit Testing for a .NET Component 


‘The fastest way to create unit testing for NET components is generate the unit test 
project from your code. This action creates the blueprint for the testing project and. 
accelerates your development. 
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Note The following sections provide a step-by-step guide to implement unit 
testing to a NET component. You can locate the full generated code in this location 
it you do not want to go through the guide 


C:\APRESS\Chapter9\BooksOrderHelperCompleted 





Follow these steps: 


|. Using Visual Studio, open the project located here: C: MPRESSV 
Chapter9\BooksOrderHelper \BooksOrderHelper.sin 


2. Open the class file BooksOrderel per. cs. 


3. Right-click at the BooksOrderHelper class level and select the 
Create Unit Tests option, as shown in Figure 9-1. 





ma 








using System; ? Lug 
E teme " 
namespace BooksOrder| $ dovinteiiy as 
t Ie thro on Cog 
[Serializable] 
public class B 
h — 
Eo med ene E 
t 1 rowan coins 
1 ak nali 


Figure 9-1. Creating the unit tests option 


4, Visual Studio shows the default properties for the unit tests. You 
can change these values if you want to create the unit test cases 
with specific names and output folders. Leave the default values 
for now, See Figure 9-2. 
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Figure 9-2, The Create Unit Tests dialog box 


5, Visual Studio generates a unit testing project and adds it to the 
solution, as shown in Figure 9-3. 





Figure 9-3. Examining the generated test project 
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6. Open the BooksOrderHelperTests.cs file and observe that Visual 
Studio has created the BaoksOrdersHel perTests class and added 
the [TestClass()] attribute: 


namespace BooksOrderHelper.Tests 


í 
[Testclass()] 
public class BooksOrderHelperTests 
{ 
[Testhethod()] 
public void returnTotalAnountTest() 
if 
Assert. Fail(); 
) 
) 
) 


7. Additionally, the test method returnTotalAwountTest() has been 
added using the [TestMethod()] attribute. 





Note The Add New Unit Test feature creates a sample test method for each 
method in the custom .Net component. 





Adding Testing Code. 
The generated code for the testing methods is Assert Fail): 


t 
[Testclass()] 
public class BooksOrderHelperTests 
( 
[Testħethod()] 
public void returnTotalAnountTest() 
( 
Assert.Fail(); 
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) 


‘The fail method forces the test method to fail, so you will need to write the custom 
code to test this method. 


To help developing clean and structured code, the AAA (Arrange-Act-Assert) pattern 
has become a standard across the development industry when it comes to testing 








projects It recommends that you divide your test method into three sections: arrange, 
act, and assert. Fach section is responsible for a small part: 


+The Arrange section initializes variables used during the method. 


+The Aet section invokes the method with the variables initialized in 
the Arrange section, 





behaves predictably. 


‘To test the returnTotalAnount method of the BooksOrdersHelper component, you 
can write two tests: 


‘+ One that verifies valid amount (<=10.000) 


+ One that sets the amount toa value bigger than 10.000. The objective 
ofthis test will be to evaluate the returnTotalAnount wh 
is bigger than 10.000, 





nthe price 


Follow these steps to create the valid amount test: 


L Open the BooksOrderHelperTests.cs Ble ofthe 
BooksOrdertelperTests .csproj project 


2. Adda reference to the BooksOrderHelper solution, 


3. Locate the test method returaTotalAnountTest(). You are going 
to use this method to validate the amount, 
4. Replace the content of the function with the following code: 
region Arrange 
int price = 1000; 
int units = 9; 
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dnt expectedValue = 9000; 
int Return = 0; 


endregion Arrange 
region act 
‘Return = BooksOrderHelper.returnTataLAnaunt(price, units); 
endregion act 
region Assert 
Assert AreEqual (expectedValue, iReturn); 
endregion Assert 


As you can see in the code: 


‘+The Arrange region sets the price, units, and the expected output of. 
the method returnTatalAnount. 

+The Act region calls the method returnTatalAnount() using the 
price and units setin the Arrange region, 





+The Assert region evaluates whether the output af the 
returnTatalAnount method returns the expectedValue using the 


statie method AreEqual of the assert object. 





Note The assert object is available for use in all testing projects. It contains a 
set of static methods that evaluate a logical condition. If this condition evaluates 
to true, the assertion passes. Otherwise, it fails. To learn about all the static 
methods that this class provides, visit https: //msdn.microsoft.com/en-us/ 
library/microsoft.visualstudio.testtools.unittesting.assert. 
aspx? f=2558MSPPError=-2147217396. 








Follow these steps to create the invalid amount test 


Ll. Copy the returnTotalAnountTest and paste it later in the 
BooksOrderHelperTests class. 


Name the new method returnTotalAmountTesthighAnount (). 


3. Add the [Testhethod()] attribute to the method ifyou did not 
copy it 
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ado 


4 


Examine the SooksOrdersHelper.returnTatalanaunt method: 
if (amount >» 10000) 


{ 
‘throw neu AxgunentException(_anount.ToString(), 
"anount not valid!"), 





) 


In the code, ifthe total amount is greater than 10.00, the. 
component will throw an argument exception. Therefore, to assert 
the output ofthe method instead of'calling the Assert object, you 
are going o add the atibute [ExpectedException typeef (Arg 
umentException))] to the zeturnTotalAnount Testiighinount 
testing method and Visual Studio will automatically assert the 
output using this ExpectedException atribute. 


Insert the following code for the 
xeturnTotalAwountTestiighAnount method. 


[TestHethod()] 
[ExpectedException typeof (ArgunentException))] 
public void returnTotalAnountTestHighAnount() 
{ 

‘region Arrange 
int price = 1000; 
int units = 20; 
int iReturn = 0; 


endregion Arrange 
region act 

iReturn = BooksOrderelper.returnTotalAnount(price, units); 
endregion act 

‘region Assert 

assertion is managed by the ExpectedException attribute, 
endregion Assert 





As you can see in the code: 


+ The Arrange region sets the price, units, and the expected output 
of the method returnTotalAnount. 


+ The Act region calls the method returnTotalAnaunt() using the 
price and units set in the Arrange region, 


+The Assert region is managed by the ExpectedException 
attribute. 


7. Choose Save Al 
3. Buld the solution. 
3. Install the BooksOrdersHelper assembly into the GAC. 
Running the Tests 
When you build the test project, the tests will appear in Test Explorer I Test Explorer is 


not shown, choose Test from the Visual Studio menu, select Windows, and then elickon 
Test Explorer. See Figure 9-4. 





Run Al | Run + | Pays Al Tests © 
Not Ran Tots 7 

© retrnTotimountent 

© reuinToalimouitfenhightimount 





Figure 9-4, Examining the tests using the Test Explorer 
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"To run all the tests, click the Run All command, as shown in Figure 9-5, 


A3 Swearing Video: Configure continuous integration 


——— 
© retunTotalimauntTestightmcant 





Figure 9-5, Running all tests using the Test Explorer 


Once the tests are complete, the Test Explorer will show the output, as shown in 
Figure 9-6 


— — 


= 
Figure 9-6, Examining the output of the test 
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Creating Unit Testing for a BizTalk Project 


In this section, you are going to use the Visual Studio Unit Testing feature to enable unit 
testing for the following BizTalk server el 





+ Maps 
+ Schemas 


+ Pipelines 





Note The following sections provide a step-by-step guide to implement unit testing 
to BizTalk Server projects. You can locate the full generated code in this location: 


C:\APRESS\Chapter9\BizTalkUnitTestingFeatureConpleted 





Adding a Unit Testing Project to the Book Orders Solution 


Follow these steps to add a unit testing project to the BoakOrders solution: 
1. Using Visual Studio, open the sample solution located here: 
C:\APRESS\Chapterg\BizTalkunitTestingFeature\, 
BizTalkUnitTestingFeatureProject\ 
BizTalkUnitTestingFeatureProject.sln 


Go to Project properties and, in the Deployment section, verify 


that Enable Unit Testing is set to True. See Figure 9-7, 
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Figure 9-7. Enabling unit testing at the project level 
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3. Choose Save All and rebuild the solution. As you enabled unit 
testing, now you have to rebuild the solution in order to tell Visual 
Studio to call the unit testing interfaces Ifyou miss this step, you. 
cannot add new testing projects to the solution, 





4, Right-click at the solution level and add a new unit test project 
(under Visual céNTest). Double-check that Net Framework 4.6 
Is selected. See Figure 9-8, 
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Figure 9-8, Adding a unit testing project to the BizTalk solution 
5. Add the following references to the project: 

+ Microsoft. BizTalk. TestTools.dl! located at C:\Program Files 
(x86)\Microsoft BizTalk Server 2016\Developer Tools 

+ Microsoft. LANG. BaseTypes. dl] located at C:\Program Files 
(x86) Microsoft BizTalk Server 2016 

+ BizTalklnitTestingteatureProject btproj which sa reference to 
the BizTalk Sample project. By doing this the testing project will have 
access to schemas, maps, and pipelines from the BizTalk project. 
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+ Microsoft. BizTalk.Pipeline.dll located at C:\Program Files 
(86) Vicrosoft BizTalk Server 2016\Microsoft BizTalk. 
Pipeline.dll 

6. Open the UnitTest.cs class and add the following using 

is, so you will be able to create objects based on the 

BizTalk project and use the Visual Studio testing features: 





using System; 
using Microsoft. VisualStudio.TestTools.UnitTesting; 
using BizTalkünitTestingFeatureProject; 

using System.10; 

using Systen.Collections.Specialize 
using Systen.Collections.Generic; 








Note All the test methods detailed in the following sections use the AAA 
(Arrange-Act-Assert) patter, so you will see this reflected by #region statements. 





Creating Test Methods to Validate Schemas 


BizTalk Server schemas have a method called ValidateTns ance that behaves in the. 





way as when you right-click the schema at design time and select the Validate. 





Instance option. In this section, you are going to create two test methods: 


+ One to validate a valid insta 





+ One to validate an invalid instance, 
Follow these steps: 
L OpentheUnitTesti.cs class and declare a string variable called 


IncomingBookOrdersInstance that will be used to save the path of 
the incoming message instance. 


public class Unitesti 
( 
[sed to save the message example instances 
public string IncomingBookOrdersInstance = ""; 
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2. Open the UnitTesti.cs class and add a Test method to testa 
valid message instance. Delete the default TestMethod and add 


this one: 

[Testhethod] 

public void ValidTestIncosingBookOrderSchema() 

{ 
Mexample test method that provides a valid message 
region Arrange 
bool boalTest0k = false; 
BizTalkünitTestingFeatureProject. InconingBaokOrders 
InconingBookOrdersSchena = new IncomingBookürders(); 
(replace the InconingBookOrdersInstance with your instance file 
InconinghookOrdersInstance = @°C:\APRESS\Chaptera\ 
BizTalkUnitTestingFeature\BizTalkunitTestingFeatureProject\ 
BizTalkunitTestingFeatureProject\NessageInstances\, 
InconingBockOrders WalidhessagesWalidIncomingBookOrders i. 
Book. xml"; 
endregion Arrange 
region Act 
boolTestOK = InconmingookOrdersSchena. ValidateInstance(Inco 
mingBookOrdersInstance, Microsoft. BizTalk.TestTools.Schema. 
(utputInstanceType.XML); 
endregion Act 
region Assert 
Assert. IsTrue(boolTest0k); 
endregion Assert 

) 


‘This TestMethod will evaluate the IncomingBookOrders schema against a valid 
message instance, 


3. Adda Test method to test for a wrong message instance: 


[TestHethod] 
public void WrongTestInconingBookürderSchema() 


dd 


{ 
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HHexanple test method that provides a wrong message 
region Arrange 

bool boolTestOk = false; 

BizTalkUnitTest ingFeatureProject -IncomingBookOrders 
IncomingBookOrdersSchena = new IncomingBookOrders(); 
I/replace the Inconing8ookOrdersInstance with your 
instance file 

InconingBookOrdersinstance = @°C:\APRESS\ 
Chapter9\BizTalidnitTestingFeature\ 
BizTalkUnitTestingFeatureProject\ 
BizTalkUnitTestingFeatureProject\WessageInstances\ 
InconingBookOrders Vironglessages 
\érongincomingBookOrders_1_Book.xnl"; 

endregion Arrange 

region Act 

boolTestOk = IncomingBookOrdersSchena.Validatetnsta 
nce(InconingBookOrdersInstance, Microsoft.8izTalk. 
TestTools. Schena OutputInstanceType. XML); 
endregion Act 

region Assert 

Assert IsTrue(boolTest0X) ; 

endregion Assert 


} 


"This Tes Method will evaluate the InconingBookOrders schema against a wrong 


message instance. 


4. Choose Save ALL 


Creating Test Methods to Validate Maps 


BizTalk Server maps have a method called TestHap that behaves the same way that 
right-clicking the map at design time and selecting the Validate Map option does. In this 
ethods: 








section you are golng to create two test n 
+ Oneto validate a map that receives a valid message instance. 
+ Oneto validate a map that receives an invalid instance. 
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The testMap method is different from the ValidateInstance of schemas: 
+ When the testHap method executes, an output fle is generated. 
+The test method does not return True or False, so you have to 

evaluate whether the output fle exists. fit does exist, that means that 


the map executed without raising exceptions. (The map could still be 
executed incorrectly from business point of view.) 





+ Checking the existence might nor be enough, so you also check thy 
‘output file is empty. In that ease, that means that the map worked 
from a validation point of view but it did not g 
because something wer 








srate any output 
‘wrong at the transformation level that did 





not raise an exception. 


Let's now implement our test method to test the Map InconingBookOrders To. 


 CRHInconingBookOrders map. 


D 


Follow these steps: 


L Adda new test method called ValidInstanceMap and type the 
following code: 


[TestHethod] 
public void ValidInstanceMap() 
( 
J[exanple test method that provides a wrong message 
region Arrange 
bool boolTestOK = false; 
Jixeplace the sDuputInstance with your output instance file 
string sOuputinstance = @"C:\APRESS\Chaptera\ 
BizTalktnitTestingFeature\BizTalkUnitTestingFeatureProject\ 
BiaTalkUnit Test ngreatureProject\Wessagelinstances\ 
Outputinstances Walidatedfapiessage l"; 
J[écleting the output instance 
if (File. Exists(sOuputinstance)) ( File. 
Delete(sOuputinstance);} 





BizTalkUnitTestingFeatureProject.Map_IncomingBookOrders_To_ 
ChiInconingBookOrders map = new Map IncomingBookrders To - 
hl nconingBookOrders () ; 
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//replace the InconingBoakOrdersInstance with your instance file 
IncomingBookOrdersinstance = @"C:\APRESS\Chaptera\ 
BizTalkUnitTestingFeature\BizTalkUnitTestingFeatureProject\ 
BizTalkUnitTestingFeatureProject\Messagelnstances\ 
TnconingBookOrders WalidMessages WalidIncomingBookOrders 1. 





Book.xnl"; 
endregion Arrange 


Hregion Act 
map.TestHap(InconingiookOrdersInstance, Microsoft.BizTalk. 
TestTols.Schena.InputInstanceType.Xnl, sOuputInstance, 
Microsoft. BizTalk. TestTools. Schena. utputInstanceType. XML) ; 
Jin this case we will check if the out put file exists 
var vFileInfo = new FileInfo(sOuputInstance); 

if (vFileInfo.Exists ) ( boolTestOK = true; ) 

endregion Act 


region Assert 
Assert.IsTrue(boolTestOk) ; 
endregion Assert 

) 


2. Adda new test method called lrongInstanceltap and type the 
following code: 


[TestHethod] 

public void WrongInstancefap() 

H 
/lexanple test method that provides a wrong message 
region Arrange 
bool hoolTest0k = false; 
//replace the sOuputInstance with your output instance file 
string sOuputInstance = 8C: MPRESSV 
Chaptero\pizTalkUnitTestingFeature\ 
BizTalkünitTestingFeatureProjectV 
BizTalkünitTestingFeatureProjectWlessageInstancesV 
OutputInstances Walidatedaplessage. xml"; 
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) 


Jleleting the output instance 
if (File.Exists(stuputInstance)) ( File. 
Delete(süuputInstance); } 
BizTalkUnitTestingFeatureProject .Map_ 
InconingBookOrders_To_CRMInconingBookOrders map = new 
Pap Incoring&ookOrders To CRMInconingBookOrders() 
/replace the IncomingBookOrdersinstance with your 
wrong instance file 

InconingBookOrdersInstance = @"C:\APRESS\ 
Chapter9\BizTalklnitTest ingFeature\ 
BizTalkunitTestingFeatureProject\ 
BizTalkUnitTestingFeatureProject\ 
Nessagelinstances\IncomingBockOrders\WtrongMessages\ 
WrongInconingiookOrders 1, Book. xl"; 

endregion Arrange 

region Act 

map. Testap(IncomingBookOrdersinstance, Microsoft. 
BizTalk. TestTools. Schena. InputInstanceType.Xnl, 
sOuputInstance, Microsoft.BizTalk.TestTools. Schema. 
QutputInstanceType XL); 

//in this case we will check if the out put file 
exists or if the File is enpty 

var vFileInfox new FileInfo(sOuputInstance); 

if (IvFileinfo.Exists || vFileinfo.Length < 4) ( 
boolTestok = false; ) 


endregion Act 


‘region Assert 
Assert. IsTrue(boolTestOk); 
endregion Assert 


3. Choose Save All. 
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Creating Test Methods to Validate Pipelines 


BizTalk Server pipeli 
that executing the Pi 
method that executes 


ines have a method called TestPipeline that behaves the same way 
ipeline.Exe tool does. In this section you are going to create a test 
ssa pipeline that receives a valid message instance. 


Follaw these steps: 


1. Addanew 


test method called FFReceivePipelineUnitTest() and 


type the following code: 


[Testuethe 


{ 


401 
public void FFReceivePipelineUnitTest() 


‘region Arrange 
[loading the flat file pipeline rcvPipBookOrdersFF 
fron the BizTalk server project 
BizTalkünitTestingFeatureProject.rcvPiphookDrderstF 
target = new rcvPipbookOrdersF(); 


[ithe testpipeline method is expecting the inconing 
message as a stringtollection// 

StringCollection documents = new StringCollection(); 
IncosinghookrdersInstance = @°C:\APRESS\ 
Chaptera\BizTallhunitTestingFeature\ 
BizTalkunitTestingFeatureProject\ 
BizTalkünitTestingreatureProject Mtessagernstancesy 
Inconinglcokrdezs Walid'essages VrFBooksOrder txt"; 
Jasert.IsTrue(File.Deists(InconingbookrdersInstance)); 
docusents Add Inconing8ookordersInstance) ; 


//0nly a body part for this test 
message so an empty collection will be 
passed, 7 
StringCollection parts = new StringCollection(); 


//The testpipeline method expects the schemas in a 
dictionary. 
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Dictionarycstring, string> schenas = new 
Dictionaryestring, string>(); 

string SchenaFile = @C:\APRESS\ 
Chaptero\BizTalhunitTestingFeature\ 
BizTalkunitTestingFeatureProject\ 
BizTalkunitTestingFeatureProject\BookOrdersConpleted. 
xsd"; 

Assert. IsTrue(File.Exists(SchenaFile)); 
schenas.Add("BizTallUnitTest ngreatureProject. 
BookOxdersConpleted’, SchenaFile); 

endregion Arrange 


region Act 
U/ons Test the execution of the pipeline using the 
inputs ===// 

target-TestPipeline(documents, parts, schenas); 

//on= Validate that the pipeline test produced the 
message ===// 

/ which conforms to the 
schena. 

#endregion Act 


region Assert 
J[Checking for the output file existence. If it exists 
that means that the pipeline executed successfully 
string[] strlessages = Directory. GetFiles(® 
APRESS\Chaptero\BizTalkUnitTestingFeature\ 
BizTalkunitTestingFeatureProject\UnitTestProject\bin\ 
Debug", "Hessage*.out"); 

Assert. IsTrue(staMlessages.Length > 0); 
BizTalkünitTestingFeatureProject. Bookürderstonpleted 
BookOrdersValidationSchena = new BookDrdersConpleted(); 
foreach (string autFile in staMessages) 


t 























tor every output file we check if the message is 
valid calling the validateInstance method. 
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Assert.IsTrue 





(BookOrdersValidationschena. 


ValidateInstance(outFile, Microsoft.BizTalk. 
TestTools. Schena.OutputInstanceType XML) ; 


) 
endregion Assert 
} 
2. Clickon Save All 


Running the Tests 


‘Torun the tests cases you created previously, just follow these steps: 


1. In the Visual Studio menu, navigate to the Test menu, select Run 
and click on All Tests. See Figure 9-9. 


Test | Analyze Window — Help 
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Passed Tests 
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Figure 9-9. Running all tests using the Test menu 


2. All unit tests will be executed. When its finished, Visual Studio 
will show the output window shown in Figure 9-10 with the test 
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Figure 9-10. Exploring the test output 





As expected, all the wrong" test methods failed. 


Performance Testing 


As discussed in previous chapters, sizing a BizTalk Server environment properly it is all 

‘about performance testing, experience, and good design principles. Thus, you should 

Include performance testing as part of your application lecycle. 
Implementing a successful performance testing requires you to deploy a testing 





environment that is nearly identical to the production environment. There are variables 
such as network, storage, Server IP and DNS names that would be different, of course, 

but hardware 
at the architecting phase because if production and testing environments are not nearly 


sizing and components should be very similar. This fact must be a priority 





Identical, you will have to extrapolate the gathered performance data in an attempt to 

represent that the test scenarios have run with production hardware resources, and thal 

‘atthe end of the day, you will generate output that is not based on real sizing. 
Additionally there are two actions that you should consider when testing BizTalk Server: 








+ Assessing the production e 





ment. 


^o Assessing the performance of individual BizTalk Server applications. 
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Performance Testing Methodology 


Every single time you plan to deploy 
this process: 
1 








new application to production y 





Assessing the performance status of the production environm 


the production performance assessment does not reveal a 
bottleneck, you can move to the stage of performing individual 

performance test for the new BizTalk application (in the testing 
environment) 

Evaluate the performance of the application against the defined 
SLA. 


the individual performance tests show that the new application 
does not reach the target SLA, you have the following options: 


à. Whenever possible, try to identify the elements within. 
the application that cause prevents the application to run. 
under the agreed performance SLA. Evaluate whether 
the application will improve its behavior by applying the 
optimizations discussed in Chapter 4 





b. Improve orchestration processing by applying the u 
explained in Chapter 6, 


niques 


Seale the production environment to fix the performance 
problem by adding more resources 


Ifthe application performs under the defined SLA, you need 
to analyze the output test data to decide whether or not the 
application will impact production performance negatively. 


that analysis reveals thatthe application can Impact the 
production environment to a point of exceeding the Maximum 
Sustainable Throughput (MST), then you should again evaluat 
whether or not to scale the production platform. 





the analysis reveals that the application will not impact the 
production environment, you can go ahead and deploy the 
application. Figure 9-11 shows this whole process. 
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Figure 9-11. Performance methodology flow 


Assessing the Production Environment 


your production environment is already running applications, itis essential that you 
know how it’s performing before you test a new solution. This information is crucial ta 
‘know the current status of the system regarding resource utilization. 

In Chapters Land 2, you learned how to interpret the most important BizTalk Server. 
performance counters and, in Chapter 3, you learned how to use the performance 
‘monitor and PAL tools to analyze the performance of a BizTalk Server platform. 

As you learned in Chapter 6, an essential consideration when planning a BizTalk 
Server environment should be to determine the maximum sustainable throughput 
(MST) ofthe system. The MST ofa BizTalk Server system is calculated as the highest 
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load of messages that the BizTalk environment can process. When load exceeds MS 





messages are que: 
Now is the time to apply all these concepts and techniques to assess the production 

environment and to evaluate whether the system is running under an optimum MST. By 

doing this you will know if there is enough room to add a new application based on the 





ed in the Message Box and transaction latency can increase. 





performance data gathered for that solution on the testing environment. 
‘To assess the production environment, follow these steps: 


1. Setup a24 hour performance log. This topic is detailed in Chapter 3, 


2. Process the captured data using PAL tool. This topic is detailed in 
Chapters. 


3, Analyze the PAL report using the thresholds and techniques 
detailed in Chapter 2 as a guide. 


A. Pay special attention to the analysis combination ofthe following 


performance counters: 
+ BizTalk: MessageBox: General Counters: Spool Size 

^o BizTalk: Messaging: Documents received-processed-sent /see 
^o Processor: Processor Time 

^o Physical Disk: Idle Time 


+ Processor: Processor time has been covered extensi 
in Chapter 2, but we need at this time 
together, 





ey 
analyze all of th 





BizTalk: Message Box: General Counters: Spool Size 





This is probably the most important or 
counter gives you an idea ofhow the syst 
analyzed period there are Increasing trends for this performance counter, there isa 


As you learned in Chapter 1, this performance 
m is processing messages. Ifduring the 





processing bottleneck that prevents the system from processing messages under the MST. 
In the other hand, ifthe Spool Size counter shows no increasing trends, that means 





that the syste 
course, if performance degradation is not observed). 


is performing efficiently and is keeping up with the current load (of 
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Physical Disk: Idle Time 


Ifthe Physical disk: Idle Time performance counter is high, especially for the SQL Server 
hosting the Message Box database, itis a good indicator that disks are processing writ 





and reads very efficiently. The disk is not busy most of the time and the loads that enter 
are being processed very quickly. On the other hand, if disks are not idle frequently 





is that they are busy and disk contention can appear. This condition can have a 
negative impact on the Message Box database, Review the section about the Message 
Bos database in Chapter 4, 


BizTall 





Messaging: Documents Received-Processed-Sent/Sec 
Under the BizTalk Messaging category, you can find these performance counters: 

+ Documents received/see 

+ Documents processed/sec 


+ Documents sent/sec. 





These performance counters by themselves do not provide any information 
related to the MST. However, if you analyze them in combination with the Spool Size 
performance counter, 
bottleneck. If the BizTalk: MessageBox: General Counters: Spool Size performance 


yu can discover the area of the engine causing the performance 





‘counter shows increasing trends during the evaluating period, check whether any of 
the messaging counte: 
scenarios, individually or combined: 





show increasing trends also, You ean experiment the following 


+ Documents Received/see inerease—That means that your 
production system is receiving more messages than usual. 
spool increases along with this counter but Documents Sent 
and Documents Processed /sec remain stable, that could means 
that the bottleneck is at the receiving layer. 


+ Documents Processed/sec increase—That means that your 
production system is processing more messages in orchestrations 
than usual. Ifthe spool increases along with this counter but 
Documents Received/s and Documents Sent/sec remain stable, that 
could mean that the bottleneck is more likely at the processing layer 
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‘+ Documents Sent/sec increase—That means that your production 
system is sending more messages than usual. If the spool increases 
along with this counter, but Documents Received/s and Documents 
Processed/sec remain stable, that could mean that the bottleneck is 
atthe sending layer because sent messages are getting suspended or 
they are retrying ports 





Example Scenarios of Evaluating the MST 


Lets review a few examples to understand spool counters versus the messaging. 
performance counters and CPU usage. This topic Is crucial when assessing the 
production environment. 


Scenario (1) Processing Messages Under the MST 





Figure 9-12 shows a performance diagram fora 24-hour capture period, 











Figure 9-12, Environment processing messages under the MST 
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‘The Spool Size counter does not show increasing trends during the whole capture. 
and Documents Received/sec does not show extreme peaks. When the number of 
received messages increases, the spool table also grows, but then it gets decreased 
‘quickly, indicating that the system is processing messages efficiently and that it can keep 
up with the current load 

"The % Processor time range forthe whole capture (36-6798) indicates that there is 
still room for receiving more messages. 

In this situation, you could add a new application if the load test in the testing 
environment does not show an excessive processor usage for that application, 


Scenario (II) Processing Messages Above the MST 


Figure 9-13 shows a different performance diagram for a 2 





-hour capture period. 








Figure 9-13. Environment processing messages above the MST 


"The Spool Size counter shows clear increasing trend during the whole capture. 
"The % Processor time range forthe whole capture (88%-100%) Indicates that the server 
is very busy processing the requests. However, Documents Received/sec does not show. 
‘extreme peaks. 
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What could be happening in this scenario? Most likely there is a processing 
bottleneck caused by orchestration processing (assuming that the sending layer is 
processing smoothly), as the documents received per second do not provide a clue. 
Now you can analyze the Documents Processed/second along with the spool size and. 
Processor time, as shown in Figure 9-14, 





Figure 8-14. Environment processing messages above the MST due to messages 
processed by orchestrations 


In situation outlined here, when the number of processed messages increases, the 
spool table also grows, and it increases exponentially along with the processed messages. 
"This indicates that the system is not processing messages efficiently and it cannot keep 
up with the load caused by messages processed by orchestrations 
Under this circumstance you cannot add a new application to the environment 
because itis already a processing bottleneck. Its time to evaluate the following options: 
+ Whenever possible, try to locate the application that causes the issue 
and evaluate whether the application will improve its behavior by. 
applying the optimizations discussed in Chapter 4. 
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+ Scale the production environment to fix the performance problem by 
adding more resources, 


+ Improve orchestration processing by applying the techniques 
explained in Chapter 6, 


Assessing the Performance of a BizTalk Application 


In previous sections, you learned how to develop unit testing projects to test the BizTalk 
Server solutions froma functionality point of view and how to assess the production 
environment, so you will now consider if there are enough hardware resources to run the 
new application. 

In this section, you are going to learn how to test a BizTalk Server solution end to 
‘end, and how to create a load test, so you can evaluate if the application is running under 
the agreed performance SLA. 

For this example, you are going to use a BizTalk application that receives messages 
using the WCF-Cusiom Receive Adapter with custom NETTCP binding. A receive port is 
listening to net. tep//localhost :8888/btsloadtest. Once the messages are published, 
asend port will write them to an output folder. 

Figure 9-15 illustrates the testing scenario. 














Figure 9-15. The testing scenario diagram. 


462 


CHAPTER) TESTING BZTALK SERVER SOLUTIONS 
"The Load Test Visual Studio project will execute, in a loop for ane 

minute, the following BizTalk Unit tests 

+ BTSMessaging 

^o BTSHessaging2 

+ BTSOrchestration 


Each loop iteration creates a new WFC message that is sent toa. 
netTcp WCF Channel. 


A BizTalk Server Receive Port is listening to that URI, 
net.tcp://1ocalhost:8888, and receives the WCF request 
through the receive location. 

"The Send port sndTestMessaging is subscribed with a filter 
expression to the ReceivePort and sends all received messages to 
the output folder. 

‘To simplify the scenario, the BizTalk application will not send a 
response back to the original WCF request 


Prerequisites 


"The coded solutions have the following requirements. 


1 


2 


Visual Studio load tests requires Visual Studio Enterprise 


‘TCP Activation, The example uses WCF Net TCP bindings and 
because of that you need a Windows service called Net TCP 
Listener Adapter. 


In Windows Server 2016, you can add the TCP Listener Adapter by 
adding the Tep Activation feature, as shown in Figure 9-16. 


463 


CHAPTERS TESTING BIZTALK SERVER SOLUTIONS 





Select features um 








E 














Figure 9-16. Adding the TCP Activation feature using the Add Roles and Features 
wizard 





Once i's installed, make sure the following Net TCP services are running: 
+ NetTep Listener Adapter 
+ Necfep Port Sharing Service 


Installing the BizTalkWCFnetTCP Sample Application 
BITNCERetTCP receives the messages from the WEC channel and sends the 
messages to an output folder. Follow these steps to install it 


L Using the File Explorer, navigate to: C:\APRESS\Chapterg\ 
BizTalkKCFnetTCPApplication 

2. Locate the following MSI file: C: \APRESS\ 
Chapters VBizTalkHCFnetTCPApplicationWSTV 
BizTalkiCFretTCPApplication.nsi 
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3. Install the MSI, 


4. Import the MSI to the BizTalk Server database using the BizTalk 
Administration Console, 


5. Start the BizTalkWCFnetTCPApplication application. 


Creating a New Visual C# Test Project 
Follow these steps to create a new test project: 


1. Using Visual Studio, create a new project. Click to expand Visual 
C$, and click Test. At the bottom of the New Project dialog box, 
specify the following options: 


* Template: Visual Cé -Test - Unit Test Project Net 4.6) 
^ Name: BizTalkUnitTest 
^^ Location: Choose a location 


+ Solution name: BizTalkUnitTest 


Click OK, 





3. Once the project loads add the following re 
+ systema 
. Systen.Servicetlodel 
. systen.Servicetlodel channels 
«o Systen.Confguration 
^o Systen.RunTine.Serial ization 


4. Add anew item to the project. Go to Visual Ce Items, navigate to 
General, and choose Application Configuration File, 


5, Double-click the App. Config fle and replace the content with the 
following code. The original file is located in 
C:\apress\Chapterg\BizTalkUnitTest \BizTalkUnitTest\ 
BizTalkunitTest. 
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66 


hid. version: 


1a 





encoding» "ut£-8"» 


«configuration 


<systen.serviceHodel> 
Bindings used by client endpoints - 
bindings» 
<netTepBinding> 
«binding names"netTcpBinding" closeTineout="01:10:00' 
openTineout»"01:10:00" receiveTineout="01:10:00" 
sendTineout-"01:10:00" transactionflow."false" 
transferHode-"Bufered" transactionProtocol-"OleT 
ansactions" hostNameConparisonMode-"StrongHildca 
xd" ListenBackloge"100" maxBufferPoolSizes"i048576" 
mmaxBufferSizes" 10485760" maxConnections»"400" 
maxReceivedMessageSize="10485760"> 











<readerQuotas maxDepth="32" maxStringContentLengthe"8192 





maxhrrayLengthe"16384" maxbytesPerRead 
‘maxNaneTablecharCount="16384" /» 
<reliableSession orderede"true 
dnactivityTimeout«"00:10:00" enablede"false" /» 
security mode="None"> 
transport. clientCredentialType- 
vel-"EncryptAndsign* /» 
<message clientCredentialType-"Mindows" /» 
</security> 
</binding> 
</netTepBinding> 
</bindings> 
client» 


096" 








<l- Client endpoints used to exchange messages with WCF 


Receive Locations --> 








«endpoint address» 





TRequestChannel" name-"BTSHessagingti 


net.tepr// dBizTalk Server Name» :8883/ 
btsloadtest" binding-"netTcpBinding" bindingContiguratio 
ne"netTcpbinding" contract»"Systen. Servicevodel. Channels. 


lindows” protectionLe 
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«endpoint address"net.tep://<BizTalk Server Nane>:8888/ 
btsloadtest" binding-"netTepBinding" bindingConfiguration 
-"netTepbinding" contracta"System. ServiceModel.Channels. 
TRequestChannel" name-"BTSMessagingkP2" /» 


<l- BTSOrchestrationEP - 

«endpoint adéresss"net.tep://cBizTalk Server Nane>:9999/ 

btsloadtest" binding-"netTepBinding" bindingConfiguration 

‘netTepBinding" contract»"Systen. ServiceMadel. Channels. 
TRequestChannel" names"BTSOrchestrationtb" /» 

</client> 








</systen. servicellodel> 
<appSettings> 
<1-- Folder containing test messages --> 
<add key-" testHessageFolder" value="C:\APRESS\Chapter9\ 
BizTalkTestingSolution\TestMessages" /> 
<add key-"ClientSettingsProvider.Serviceuri" values" /> 
</appSettings> 
configuration. 








"This is the configuration file where you set the netTepBinding properties and the 
location of the test messages used by the test case: 


+ netTepBinding properties Send messages through using 
the netTCPbinding on ports 8888 and 9999. Since our receive 
locations will be listening on those ports, BizTalk Server will pick 
up messages from there. 


+ Location of the test messages—Locate the test message folder 
using the TestMessageFol der key 


6, Visual Studio created an empty UnitTest1.cs file. Open itand 
replace the content with the already generated class, located here: 


C:\APRESS\Chapter9\BizTalkUnitTest\BizTalkUnitTest\ 
BizTalkunitTest\unitTest1.cs 
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‘This class implements the load test interfaces that will be called by 
Visual Studio test engine. 

7, Inthe Visual Studio menu, navigate to the Test menu, select Run, 
and click on All Tests See Figure 9-17. 





Test | Analyze — Window — Help. 
Run 





A Sdeded Tests 
I Debug ^ PS All Tests 
Poyis e | Fated Tens 
Oo messis ^| — Not Run Tests 
Analyze Codi Coverage > | passed Tests 
J— 


| Repeat Last Run — CURL 


Windows 


Figure 8-17. Running all tests using the Test menu 
3. Choose Save All. 


Adding a Load Test to the Project 


Now it's time to simulate a load against the united test you created previously. Follow 
these steps to add a load test to the project: 


L. Create a new folder called LoadTests. You will use this folder to 
save the LoadTest cases data. 


2, Right-click at the LoadTests folder level, select Add > New Item, and 
choose the Load Test option under Visual c items Test. The Create 
New Load Test Wizard window will be shown, See Figure 9-18. 
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Figure 9-18. Selecting the on-premise version of the load test 


3, From the welcome screen, select On-Premise Load Test and. 


click Newt, 
On the Review and Edit Run settings fora load test window, take 
the following actions (see Figure 9-19): 
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i . 









































Figure 9-19. Selecting the load test duration mode 
^o Select the Load Test duration option. 
^ Specify 30 seconds for the Warm-Up Duration setting. 


^ Change the Run Duration to 1 minute or leave it at 5 (five minutes 
vill load more than 4,000 instances into BizTalk Server), 


^ Set the Sampling Rate setting to 5 seconds. 
^ Invoke validation rules marked Low as the Validation level. 
5. Click New. 


6. On the Edit Settings for a load test scenario, type TestiCFletTcp 
as the name ofthe scenario and choose the Do Not Use Think 
‘Times option. See Figure 9-20. 
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Figure 9-20. Entering the name of the load test scenario 


7. Click Next 


3. From the Edit Load Pattern settings window, choose Step Load 
and assign the following values to configuration step settings 
(see Figure 9-21a): 


^ Start user count: 10 users 
+ Step duration: 60 seconds 
^ Step user count: 10 users 


^ Maximum user count: 80 
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Figure 9-21a. Setting the load pattern for the load test 


9. Click Next 


10, From the Test Mix Model window, select Based on the Total 
Number Of Tests and click Next, as shown in Figure 9-21b, 
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Figure 9-21b. Setting the mix model for the load test. 


11. You now select the unit test eases you ereated previously. From 
the Add Tests window, click Add. Then add the following test cases 
to the scenario: 


^ BTSMessaging 
. BTSHessaging2 
+ BTS0rchestration 


12, Use the sliders to set the BTSMessaging unit test to a 60% 
distribution. See Figure 9-22. 
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Figure 9-22. Selecting the unit test scenarios and load distribution 


13. Click Next. 
14, Leave the LAN settings at the defaults and click Next, 


15. From the Specify Computers to Monitor window, add Localhost 
In the nest section you will be adding BizTalk Server performance 
counters, so you do not need to add counters at this stage. 


16, Click Finish. 


Create a BizTalk Server Counter Set 


While the load test runs, you should capture all the BizTalk Server relevant counters 
Tor that scenario, Visual Studio does not come with a counter set for BizTalk Server. 

ju will have to create one. Use the counters referenced throughout all the 
book, especially in Chapter 2, 





Therefor 
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Follaw these steps 


1. Using the Visual Studio Solution Explorer, double-click the 
LoadTest1. loadtest file. See Figure 9-23. 
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Figure 9-23. Opening the load test file 


Locate the Counter Sets node. Right-click and select the option. 
‘Add Custom Counter Set. See Figure 9-24. 
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Figure 9-24. Adding a custom counter set 


3. Notice that an empty Customl CounterSet is created. 


Right-click the Custom! counter set, select the Properties option, 
and change the name to BizTalk Server. 


Locate the new BizTalk Server counter set, then right-click and 
select the Add Counters option. See Figure 9-25. 
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Figure 9-25. Adding counters to a counter set 


6. 


Add the following BizTalk Server performance counters: 


Processor: Select only Privileged Time and %Processor Time 


BizTalk: Message Box: General Counters: Select all counters all 
instances. 


BizTalk: Messaging: Select all counters, all instances 
BizTalk: Message Agent: Select all counters, all instances 


XLANG/s Orchestrations: Select all counters, all instances 
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As a result, you should see something like Figure 9-26. 
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Figure 9-26. Exploring the added performance counters 


7. Right-click at the Counter Set Mappings root level and select the 
Manage Counter Sets option. Make sure that only BizTalk Server is 
selected and lick OK. See Figure 9-27 
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Figure 9-27. Accessing the counter set mappings 


am 


CHAPTERS TESTING BIZTALK SERVER SOLUTIONS 


3. Check the BizTalk Server Counters checkbox and click OK. See 
Figure 9-28, 
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Figure 9-28. Selecting BizTalk Server performance counter set 
9. Choose Save All. 

Running the Load Test 

Lets run the load test: 


1. Open the LoadTest1. oadtest file. 


2, Right-click at the root level and choose Run Load Test. 
See Figure 9-29, 
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Figure 9-29. Running the load test 


3. Theload test will now enter the warming up phase. Visual Studio 
will show the remaining warming up time, as shown in Figure 9-30. 





Warn Up Remaining: 0010 MN 


Figure 9-30. Showing the warming up progress 


4, Wait until warming up time finishes. Visual Studio will run the 
load test scenario. It will show you the remaining time, as shown 
in Figure 9-31. 


eins 0005 MN 


Figure 9-31. Showing the remaining time for the load test 


5. Once t's finished, Visual Studio will show the Load Test Summary 
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Examining the Load Test Results 


Let's now review all the data generated by the load test, This information is crucial to 
understanding whether the new application will affect production performance. 





Note Keep in mind that, along with the load test output data, you could also set 
up a performance log using the performance monitor tool and then analyze the 
data by running the PAL tool. You can then include the PAL report as part of the 
load test result by copying the PAL output files to the test folder. 





Load Test Summary Dashboard 


‘The Summary dashboard is accessible from the Load test menu, as shown in Figure 9-32. 





Figure 9-32. Accessing the load test summary section 


Generated data: 
1. General information: Overall Test Run Information, Max User 
Load forced, Tests Run Per Second, Number of Failed Tests, and 
the Top 5 slowest tests, See Figure 9-33. 





Figure 9-33. Examining the load test summary data 
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2. The overall results are shown in Figure 9-34, 


Overall Results 
Max User Load 10 
Tests/Soc as 
Tests Failed 2 
Avo. Test Time (see) 030 
Transactions /Sec o 
Avi. Transaction Time (sec) o 
Pages/Sec - 
Avo. Page Time (sec) E 
Requests/Sec 
Requests Failed - 
Requests Cached Percentage. E 
‘Aug. Response Time (sec) - 
‘Avg. Content Length (bytes) - 


Figure 9-34. Examining the overall results data 
3, For each unit test, Visual Studio shows the number of executed 


tests, number of failed tests, and the average individual test ime 
In seconds, as shown in Figure 9-35. 
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Figure 9-35. Examining test results data 


4. also shows overall physical resource consumption — Processor 
Time and Available Memory—as shown in Figure 9-46. 
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Figure 9-36. Examining the resource utilization 


5. also shows the errors that occurred during the load test 
execution, grouped by error type, as shown in Figure 9-37. 
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Figure 9-37. Exploring the list of errors generated during the test 


Graphs Dashboard. 


Graphs view is accessible from the Load Test menu, as shown in Figure 9-38. 
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Figure 9-38. Accessing the graph summary section 


Generated data: 


1. In the Default Graph view, there isa pane on the left that shows all 
the counter categories you have access to, as shown in Figure 9-39. 
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Figure 9-39. Exploring the graph information 
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"This view is dynamic, so if you select a diferent category, the graphs and 
performance counters views will change accordingly. 





Note Notice the Error symbol at the Counters Category Computers. This icon 
appears when any ofthe studied performance counters are not within the default 
thresholds. 





2, Navigate to Computers\<BizTalk Server Name>\MessageBox 
General Counters. Locate the Spool Size counter, right-click, and. 
select Show Counters on Graph option. See Figure 9-40, 
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Figure 9-40. Accessing the Spool counter data 
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‘The graph view will change to display the Spool Size performance counter data, as 
shown in Figure 9-41. 

























Figure 9-41. Examining the Spool counter data 


Generating an Excel Report 


"This feature allows you to crate Excel reports with the analyzed data, so you will be 
able to share the output results. This feature requires Microsoft Excel installed in your 
computer. Otherwise, it will not be available, 
Follow these steps to create an Excel report 
1. Athe Load Test results, click on the Generate Excel Report 
toolbox icon, as shown in Figure 9-42. 
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Figure 9-42. Accessing the Excel exporting tool 
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"The generate report wizard will start. 
3, Choose Create a Trend Report and click Next 


4. Type the following name for the report: BizTalk Trend Report, 
Click Next. 
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5. Select the latest load test results. Click Next. 


6. Under the Message Box Agent category, select only the following 
performance counters: 


^ Message Delivery Throttling State 
+ Message Publishing Throttling State 

7. Under the Processor category, select only the Processor Time 
performance counter 

3. Under the BizTalk: Message Box: General Counters category, 
select only the Spool Size Counter 

9. Under the Memory category, select only the Available Mbytes 
performance countes 

10. Cliek Finish. Visual Studio will generate the Output Report. It 
should look something like Figure 9-43, 





Name: BizTalk Trend Report 
Table of Contents 


Runs in report 
Rans 


Reports 
'BiTalkcMessage Agent 
Mesi delery throng state 
Message publishing throttling state 
— Box-Genaral Counters 


Spool sve 
Memory 
Processor 

S Processor Time 


Figure 9-43. Examining the Excel report 


1L. Navigate through the report to access the performance data. 
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Creating a Web Test for BizTalk WCF Service HTTP Binding 


In this section, you are going to learn how to test a BizTalk Server application that has an 
the application is running 








orch 





tration exposed as a WCF service, so you can evaluate 
under the agreed performance SLA. 

In this example, a BizTalk application called WebServiceBookOrders receives 
messages using the WCF-BasicHttp receive adapter. 

Follow th 








se steps; 
1. Installand import the WebServiceBookOrders MSI file 
located here: 
C: MPRESSAChapter9 WiebServiceBookOrders WST 
2. The MSI file will ereate a WCF service called 
WebServiceBookOrders and a BizTalk Server application with the 
3. Using Internet Information Server, change the application pool to 
BAMAppPool, 


4. Start the BizTalk application called iebServiceBookOrders, 
Otherwise, the WCF s an fall. 





Once it's installed, use your preferred Internet browser to navigate 
to the following URI: http: //localhost /WebServiceBookOrders/ 
WebSexviceBackOrders orcBookOrders Port _WebBookOrders.svc 





Warning Note that troubleshooting IIS and WCF services are out of the scope 

of the book. If you do not see the service description and the WSDL, the steps will 
fall, Troubleshoot your local Internet information server until you are able to see the 
service definition. 





6. Using Visual Studio, create a new web load test project. 
See Figure 9-44. 
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Figure 9-44, Adding a web performance and load testing Visual Cè project 











7. Visual Studio creates the test project and adds an empty web test 
3. Double-click the ebtest1 webtest file. 


9. Right-click atthe web test level and select the Add Web Service 
Request option. See Figure 9-45, 
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Figure 9-45. Adding a web service request 





10, Visual Studio creates a default local host request, 
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1L. Click on the default local host request and, in the Properties 
window, set the URL property wohttp://localhast/ 
WebServiceBookOrders/WebServiceBookOrders orcBookOrders_ 
Port HebBookOrders. svc, See Figure 9-46, 
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Figure 9-46, Setting the destination URL property of the web test 





12, Right-click at the test evel and select the Add Header option, as 
shown in Figure 9-47. 
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Figure 9-47, Adding a header to the web request 
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13. Visual Studio adds a header section and a body string parameter 
that will be used to send the request to the service. 


14, Expand the Headers folder, select the default header, and right- 
click and select Properties, See Figure 9-4, 
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15. Setthe Name field to SoapAction. Set the Value field based on 
your web service action name, The SoapAction value can be 
‘obtained from your web service's WSDL. In this case the BizTalk 
Server application exposed the service with WebBookOrders as the 
SoapAction. See Figure 9-49. 
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Figure 9-49. Setting up the SoapAction for the service request 
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16. Right-click on String Body and choose Properties. See Figure 9-50, 
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Figure 9-80. Accessing the properties of the body string request parameter 


17. Settext/xml to Content Type. Set a valid request XML into the 
string body. In this case, I generated the request using the Soap. 
UI, but you can leverage any other software for that objective: 





<soapenv:Envelope xnlns:soapenv="http://schenas..xnlsoap. 
org/soap/envelope/" xnlns:web-" http: / /iebServiceBaokOrders.. 
BookOrderRequest"> 

<soapenv:Header/> 

«scapenv: Body» 

«eb: BookOrderRequest> 
<OrderDate>21/04/1977</OrderDate> 
<Customerld>777888/Custonerid> 
<Total Anount>345666¢/Total Anount> 
 dDescriptionteste/Description, 
<isVIP>O0c/isVIP> 
<Order1d>453453433</0rder1d> 

«ueb: BookOrderRequest> 

«/soapenv Body» 
</soapenv:Envelope> 


18, Choose Save AIL 





19. Click the Run Test button (see Figure 9-51) to test your service call. 
Make sure that it is successful and fixit if the test failed, 
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Figure 9-51. Running the test 


20. You should geta response from the service, as shown in Figure 9-52. 





———— 
































Figure 9-52. Examining the response from the service 
2. Nowyou can add a load test to run the web test. Follow the steps 
detailed in the previous section, entitled “Adding a Load Test to 
the Project: 
Providing a Dynamic Data Source 


For most ofthe performance testing labs, you should include dynamic data sources that 
will help you test all possible scenarios for the business flow. 
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At the moment of writing this book, Visual Studio provides the following options to 
retrieve data from a dynamic data source 


+ Database 
+ csviile 
. XMLale 


Follow these steps to add a CSV file as the data source for a load test project 


1. Select the web test and click the Add Data Source button. See 
Figure 9-53, 





Figure 9-53. Adding a data source to the web test project. 
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2, Visual Studio will oad the wizard for data source selection. Type 
BookOrders as the data source name, select CSV file as the data 
source type, and click Next. See Figure 9-54. 
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Figure 9-54. Selecting the type of the data source. 
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3, Select Cc MPRESS Chapters LoadTestiebServiceBookOrdersV 
DataSourceWiebServiceBookOrdersRequests. csv as the CSV fle 
and click Finish. See Figure 9-55. 
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Figure 9-55. Selecting the CSV file and exploring the data 
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4. Click Yes if Visual Studio asks you if you want to include the CSV 
fle as part or the project. See Figure 9-56. 
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Figure 9-56. Including the CSV file as part of the testing project 





5. Right-click the WebServiceBookOrdersRequeststcsy table and 
select Properties. See Figure 9-57. 
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Figure 9-57. Selecting the table properties 
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6. Change the Access Method to Random so that Visual Studio will 
randomly access the CSV file, See Figure 9-58. 
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Figure 9-58, Selecting the Access Method 


7. Right-click at the String Body section of the web test request and 
click on Properties. See Figure 9-59. 
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Figure 9-59. Accessing the properties of the string body request property 
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E 


Edit the string body property to the following and replace it with, 
the following text: 


<soapenv:Envelope xnlns:soapenv="http://schenas.xnlsoap. 
org/soap/envelope/" xnlns:weh-" http: //iebServiceBookOrders. 
BookOrderRequest"» 








cweb:BookOrderRequest> 


<OrderDate>{ (BookOrders.WebServiceBockOrdersRequestsécsv. 
OrderDate}}</Orderdate> 


<CustonerTd>{(BookOrders. ebServiceBookOrdersRequestsiics 
v.Custover1d))«/Custonerld» 


‘<TotaLAnount ((BookOrders. WebServiceBookOrdersRequeststcs 
v.TotalAnount}}</Totalénount> 


«description TestDescriptionc/Description, 
<ASVIP» {(BookOxders. ebserviceBookürderstequestsscsv. 
‘isVIP}}</isVIP> 


<OrderTd{(BookOrders. WebServicetookOrdersRequestsicsv.Orderls)) 
</orderié> 


</web: BookOrderRequest> 
«</soapenv :Body> 
</soapenv:Envelope> 


Notice that instead of providing a fixed value, you are now 
retrieving the data from the CSV file, using the format 
{(DataSourceNane. TableNane. ColumNane)) for instance in our 
example. Ifyou want to provide the Customerld parameter, you 
should use this format 


{{BookOrders.WebServiceBookOrdersRequests#csv. Custonerld]) 


Now run the test several times and you will see how Visual Studio is 
using data from the CSV fle. 
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Summary 


In this chapter, you learned how to use all the knowledge acquired in previous 
‘chapters of the book to include testing procedures as part of your normal application 
development lifecycle 

Performance testing is very important for BizTalk Server environments as usually, 
BizTalk Server integrates applications that are part of the core business within the 
organization. That is why implementing the unit and performance testing techniques 
discussed in this chapter will increase the quality of the code you write. It does this by 
reducing the chances of bugs and ensuring that the production environment will be able 
to keep up with the load that new applications will generate. 

You also learned about the preferred performance methodology that mature 
‘customers are applying to their mission-critical BizTalk Server environments. 

In Chapter 9, you read a BizTalk Server tale that will help you put into practice 
several advanced topics discussed in this book. 
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CHAPTER 10 


A BizTalk Server Tale 


In this chapter, Lam going to tell you a story. This is the story about a fictitious customer 
(ACME corporation) that decides to start with BizTalk Server as its integration system. 

Because at the beginning u 
the sizing of the initial environment is small, Gradually th 





start only with one non-mission critical application, 
add more and more 





applications, so they have to igure out a way to improve the platform. Even though this 
is an imaginary exercise, the situations described here are quite common with each 
evolving BizTalk server customer, 

Additionally, you will earn about the most common architectures that customers 
are using all over the world and how they provide high availability to the single point of 
failure areas. 


Chapter 1: The One with the Baby BizTalk Server 


ACME corporation started using BizTalk Server to move customer data from an Excel 

fle to the internal CRM application. Ifthis solution fails, business operations will not be 

affected, as this application is mainly used for internal reporting; the core solutions are 
id appli 

use of he light availability and performance requirements, they decided to start 

'andard edition, which has 


currently operated by custom NET services ions, 
Bei 
‘with a small BizTalk Serv 


the following limitations: 











deployment using BizTalk Serv 





. Access to CPU is restricted by two cores 
+ Only one BizTalk Server İs possible 
^ Maximum of five BizTalk Server applications 


As they do not need high availability, BizTalk and SQL Server are on the same box to 


reduce hardware and licensing costs 


© Agustin Mine m 
A Måntaras, Tak Serer 2016 pedo or 0 LU 711-2389, 16 





CHAPTER id A BIZTALK SERVER TALE 


Archivectu 





+ Number of total servers: 1 
+ Number of SQL Servers: 1 
+ Standard edition, 2 CPU, 8 GB ofm 





+ Number of BizTalk Servers: 1 
+ Standard edition, 2 CPU, 8 GB ofm 








Chapter 2: The One with New Applications 


After using BizTalk Server for a while, the team saw the great value of the mapping 
Feature and how BizTalk Server could convert fat files to XML. The develops 
‘wanted ro create more applications and deploy d 








10 BizTalk Server. However, they 
realized that they would need to upgrade the BizTalk Server version to the Enterprise 
edition because now th 





‘would need eight applications to provide the business. 
functionality. Additionally, upon testing, they realized that the CPU was above 70% most 
ofthe time, so they decided to scale up the server to four cores. SQL and BizTalk are stil 
‘on the same standalone serv 








New Architecture: 
^o Number of Total servers: 1 
+ Number of QL Servers: 1 
^o Standard edition, 4 CPU, 8 GB of memory 
^o Number of BizTalk Servers: 1 





xerprise edition, 4 CPU, 8 GB of memory 





Chapter 3: The One with Performance Problems (I) 


After a few months, the DBA observes that the SQL Server process is using high CPU. 
andis preventing the BizTalkServerApplication host instance from accessing CPU 
resources. Applications are slow. To fs this problem, the team decides to move SQL. 
Server toa dedicated server. 





CHAPTER id A BIZTALK SERVER TALE 


After some testing in the testing environment, they observe that the SQL Server 
process CPU consumption i still high, but the BizTalk Server host instance performs 
smoothly, so they decide to apply this new architecture to production. 

The team will now monitor CPU consumption in both servers. 

New Architectu 








. Number of otal servers: 
+ Number of SQL Servers: 1 

+ Standard edition, 4 CPU, 16 GB of memory 
^o Number of BizTalk Servers: 1 


+ Enterprise edition, 4 CPU, 8 GB of memory 


Chapter 4: The One with Performance Problems (II) 


One of the most commonly used applications starts processing more messages than the 
original performance definition. BizTalk Server administrators now observe that CPU 
‘consumption during business hours increases to 90%, so they decide to add more CPU 
resources to the BizTalk Server. 

Upon testing, they realize that even by adding more CPU to the BizTalk Server, they 
are notable to increase the rate. After further investigations they observe that SQL Server 
CPU usage is also increasing, so they decide to number of CPUs that SQL. 
Server utilizes. After this change, the platform runs efficiently. 

‘The team will now monitor CPU consumption in both servers. 

New Architectus 








icrease th 








+ Number of otal servers: 
+ Number of QL Servers: 1 

+ Standard edition, 8 CPU, 8 GB of memory 
. Number of BizTalk Servers: 1 


+ Enterprise edition, 8 CPU, 8 GB of memory 
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Chapter 5: The One with the Disaster (1) 


Over weekend, for some unknown reason, the SQL Server stops working. This causes 
‘one of the most important BizTalk S 





ver applications, which is preparing data for the 
working week, to not process files on time. The business operations were therefore 
delayed until the DBA could fix the issue on Monday. The SQL Server was down due to a 
failure related to the local disks. 

"The DBA suggests to the project manager that ifthis application had become that 
important, it might be time to add a second server and provide high availability tothe 
‘SQL Server layer by implementing a Microsoft cluster. 

The project manager approves the budget and the team implements a duster in an 
active/ passive configuration with only one SQL Server instance. 

New Architecture: 


^ Number of otal servers: 
+ Number of SQL Servers: 2 ina cluster configuration 


+ Standard edition, 8 CPU, 8 GB ofm 





ny 
+ Number of BizTalk Servers: 1 


+ Enterprise edition, 8 CPU, 8 GB of memory 


Chapter 6: The One with the Disaster (Il) 


This time, around 2AM during a normal work week, BizTalk administrators receive alerts 
because the environment is not processing anything. They go onsite, and they detect 
that the BizTalk Server is down. Upon investigation, they notice the sam 
‘caused SQL Server to stop the previous month is appe 
local BizTalk Server disk. 

As there is an application that, during the night, processes important information for 
the business, the BizTalk Server administrator suggests that providing high availability to 
the BizTalk Server layer will ensure that this issue will not happen again. 

‘The project is approved again by the project manager and the team adds another 
BizTalk Server box. 

Upon testing, they realize that the platform can process even more messages than 
‘before, and that BizTalk Server CPU consumption has decreased by 20%. 


disk issue that 
ing again, but this time in the 
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The DBA raises a warning, because overall SQL Server CPU usage has increased by 
This is because there are now two host instances polling the databases. 
New Architecture: 





^o Number of total servers: 4 

+ Number of QL Servers:2 in a cluster configuration 
+ Standard edition, 8 CPU, 8 GB of memory 

^o Number of BizTalk Servers: 2 


+ Enterprise edition, 8 CPU, 8 GB of memory 


Chapter 7: The One with the Big File that Changes 
Everything 


A new application comes onto the scene. This application needs to process a 1 GB fat 
d the output to the CRM application. 
start coding the solution with a small sample fle that contains all the 
possible message definitions. 


fle daily, apply some transformations to it, and s 








Developer 


When they test the solution with a real file, BizTalk Server starts processing messages 
‘extremely slow, to the point thatit takes more than five hours to complete the whole 
process. Additionally, while the system is busy processing the flat file, the rest of 
applications are processing extremely slowly. 

As they do not know what is happening they decide to open a Microsoft Support 
‘ease, After a few days of investigation, the support engineer comes back with the 
following explanation: 


^ BizTalk is entering a throttling state due to memory pressure and that is 
most likely the reason why the process is taking so long to complete. 





+ “The rest ofthe applications do not have enough resources to process 
messages atan acceptable rate, because orchestrations and receive 
host, and when the large 
(consuming most ofthe server 
resources) in the disassembling stage. 


and send ports are running under the san 





file comes, the host instance is bus 





+ The XML represe 
memory to the BizTalk Server process. 


{ation ofthe flat fle allocates around 8 GB of 
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‘To fix this problem, the engineer recommends the following plan: 


Create a set of hosts for receiving, processing, and sending, and have 
the non-large lat fle applications run under these new hosts 


Create a new set of hosts dedicated exclusively to process the large 
fle and increase the process virtual memory threshold to 50% so 
that all the host instances of this application can take up to 50% of 
available memory on the server. 








Increase the BizTalk Server memory so it will have enough resources. 
to process the large file. 

Isolate the DTC log fle into a dedicated disk (BizTalk and SQL 
Server). 


‘The BizTalk team at ACME corporation implements all the suggested changes and, 
upon testing, they realize that the process completes in a reasonable period, from the 
business point of view. Additionally, the rest of the applications are still processing 
messages without entering a throuling state. 

From now on, the team will consider the host separation policy as an important 


subject. 


New Architecture: 
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Number of total servers: 4 
Number of SQL Servers: 2 in a cluster configuration. 
+ Standard edition, 8 CPU, 32 GB ofn 





Number of BizTalk Servers: 2 

+ Enterprise edition, CPU, 32 GB of memory 

‘There is nowa new set of hosts (receiving, processing, and sending) 
that process the large-file application, 


“Theres also a new set of hosts split by BizTalk Server functionalities 
to deal with the rest o'applications. 
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Chapter 8: The One with Web Service 


At this point, BizTalk Server applications were receiving messages only from shared 
folders over the network. ACME corporation was changing fast, though, and was 
evolving to a more service-oriented architecture. Part of this transformation creates the 
need to expose several applications as rest services. 

Developers attend a Microsoft course and they learn the right steps to publish 
orchestrations as Web Services. However, when they test the solution, they realize that 
only one BizTalk Server is receiving all the requests. The CPU on that server rises to 80%, 
while the other one stays stable with an average consumption of 20%. They contact the 
networking team and together they come to the conclusion that they need to implement 
a network load-balancing mechanism that will distribute the load across both BizTalk 
Servers. 











New Architecture: 
+ Number of otal servers: 4 
+ Number of QL Servers: 2in a cluster configuration 
+ Standard edition, 8 CPU, 32 GB of memory 


^ Number of BizTalk Servers: 2 impl 





nented using a Hardware NLB 
solution to distribute the load 


+ Enterprise edition, 8 CPU, 32 GB of memory 


+ There is now a new set of hosts (receiving, processing, and sending) 
that are processing the large-file application, 


+ There is also a new set of hosts split by BizTalk Server functionalities 
to deal with the rest of applications 


Chapter 9: The One with the Disk Performance Issue 


Developers a ng applications and deploying them into production on a monthly 
basis. With the new architecture, no performance issues have been detected and the 
platform is up and running efficiently 
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One afternoon, the DBA observed that SQL Server disks are performing very 
slow and that the read and write latency for the disk holding all of the BizTalk Server 
databases has increased to the point that, on average, write and reads operations are 
taking more than 25 milliseconds. The DBA contacts the storage team and they discard 
‘configuration problems on SAN level, as they are following the vendor best practices. It 
Just seems that the disks holding the BizTalk Server databases are quite busy, 

At this stage, several business areas start complaining about the performance of the 





BizTalk Server applications because they observe that overall throughput is affected, 
The DBA raises the issue and recommends that they separate data and transaction 
Iles into different LUNS in the SAN. The storage team then dedicates several LUNs to 
allocate all the required disks to BizTalk Server. 
‘They come up with the recommendation outlined in Table 10-1 


Table 10-1. Data. 
Database File Drive 





les Distribution 


Messagetox data 
MessageBox log 


BizTalk tracking data 
BizTalk Tracking log 


BizTalk BAM primary import data 
BizTalk BAM primary import log 
Business application data. 


Rest of databases data 


4 
K 
L 

Business application log m 
n 

Rest of databases log o 


Atter the team applies the changes, the disks performance tum back to acceptable values. 
New Architecture: 


^ Number of otal servers: 
+ Number of SQL Servers: 2 ina cluster configuration 


+ Standard edition, 8 CPU, 32 GB ofn 
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+ Number of BizTalk Servers: 2 implemented using a Hardware NLB 
solution to distribute the load 
+ Enterprise edition, 8 CPU, 32 GB of memory 


‘+ There is now a new set of hosts (receiving, processing, and sending) 
that are processing the large-fle application, 

+ There is also a new set of hosts split by BizTalk Server functionalities 
to deal with the rest of applications, 





+ Anew disk infrastructure is provided that guarantees BizTalk Server 
database isolation. 


Chapter 10: The One with the New Application (I) 


‘A new business application comes that requires processing 500 messages per second 
during the entire day. Developers develop the application and when they test itin the 
testing servers, they realize that they are not getting even close to that number and the 
available memory on BizTalk and SQL Server decreases into the danger zane, 

‘To fix the issue, the BizTalk administrator recommends they create a new set of hosts 
Tor receiving, processing, and sending and dedicate them to the new application so the 
team can test this separately and adjust all of the required performance values 

After seve 





I days of testing, the team comes to the following conclusions: 


+ Pooling intervals—Reduce to 50 milliseconds for all of the 
application hosts. By doing so, new messages and orchestrations are 
processed earlier by the engine (almost as soon as they are assigned 
to the host queues). 

+ Disabling orchestration dehydration If failure occurs in the 
middle ofa Web Service call, the consumer ofthe service will send 
the request again so there is no need to dehydrate orchestrations. 





+ As dehydration is disabled, orchestrations will not be stored in the 
MessageBox while they are consuming a service; they will be held on 
memory. BizTalk Servers now have to have more memory to keep up 
with the load. 
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+ Maximum engine threads—By increasing this setting to 40 it seems 
that the BizTalk engine can increase the rate of documents processed 
per second. However, SQL Server CPU rises exponentially, so they. 
decided that they have ta scale up SQL Server by adding more CPUs, 





+ NET CLR host instance settings—They also observed that the 
receiving and sending host instances are processing more messages i 
they decrease the value of the maximum TO threads to 100. 





With all hese customizations, they reach a rate of 400 messages per s 
platform enters in throttling state because of the database size. 

They tune then the Message Count in Database setting to 100,000 so that the engine 
‘enters the throttling condition later As SQL Server needs to process more messages, 
they contact the DBA team to ensure there is enough backlog for the drives that hold 
BizTalk Server databases. The DBA evaluates the new situation and changes the SAN 
distribution to add more space to the BizTalk Server database drives. 

Developers now test the solution and they get very close tø the performance 
requirement of 500 messages per second. However, DBA detects that the MessageBox 
database ls growing very fast and disk latency increases exponentially over 30 
milliseconds. 


nd, but the 





Alter sev 





al hours of investigation, they do not 
decide to open a support case. 

Once the engineer reviews all the data, analyzes all the BizTalk traces, and runs 
several troubleshooting tools, he delivers an explanation and an action plan: 


root cause of the Issue, so they 


+The reason that the platform is slow is because the MessageBox: 
database has become a bottleneck. To fix this situation, you should 
test the solution with the following changes: 





+ Add two more message boxes to distribute the message 
publication between them, 


+ Isolate all MessageBaxes to run in a separated SQL Server 
instance in a dedicated server. 


‘The team implements all the suggested changes in the testing environment. The 
solution can now achieve the performance target of 300 messages per second while the 
latency is stable for the rest ofthe applications. SQL disk performance is not an issue. 
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New Architecture: 





+ Number of total servers: 4 


‘+ Number of SQL Servers:2 ina cluster configuration. MessageBaxes 
are running on server A and server B has the rest of databases, 


+ Standard edition, 12 CPU, 64 GB of memory 


+ Number of BizTalk Servers: 2 implemented using a Hardware NLB 
solution to distribute the load. 


+ Enterprise edition, 12 CPU, 64 GB of memory 


+ There is now a new set of hosts (receiving, processing, and sending) 
that are processing the large-fle application, 


+ There is also a new set ofhosts split by BizTalk Server functionalities 
to deal with the rest of applications 


Chapter 11: The One with FTP Server 


‘A new application that needs to retrieve files from a FTP server arrives. Developers 





reate the application and test it. Everything looks fine, so the solution goes live A few 
‘months later, the business responsible for that application complains that from time to 
time they receive duplicated requests. 

Developers analyze the code, looking for bugs, but they cannot find anything. Using 
the BizTalk administration console, they detect those duplicated message Instances 





by enabling message body tracking, but they cannot still figure out why the duplicated 
messages are being generated. They seriously consider that there might be a bug in the 
product, They install the latest BizTalk Server 2016 service pack, but as the issue is still 
there, they decide to open a support case. 





During the initial call, the engineer comes up with a solution right away: 


+ The FTP protocol does not block the file while a process is accessing 
i It is not like an NTFS folder. This is by design and this behavior is 
out of the scope of BizTalk Server. In this ase, as you have two host. 
instances accessing the FTP server, it might happen that eventually 
both processes will access the same file at the same time. Th 





eive 





location will piek up those messages and the engine will publish 
them as if they were different messages, 


CHAPTER id A BIZTALK SERVER TALE 
‘+ Therefore, there are two options: 


a. You can disable one host instance from starting on one of the 
servers. With this action, you will ensure that no duplicated 
messages are published, but you will ose high availabilty for the 
FIP host, 


b. You can cluster the receiving FTP host, so only one host instance 
{sstarted ata time, However, you are using NLB and clustering 
at the same time is not supported. To solve this situation, you. 
have to add. 





(o BizTalk Servers to the group and cluster them. 
Then you could dedicate those servers to run only the receiving 
FTP hosts. These two new servers will be outside of the NLB. 





The ACME team evaluates the options and, as this application i not mission-critical 
and there are budget restrictions at this time, they decide to create a new host and put 
the FTP receive location under it, They will start the associated host instance on server A 
and they will disable the host instance on server B. Ifthere is an issue with the running 

nee, they will manually start the second one and they will deal with all the 
associated issues reactively. 





hostins 





Chapter 12: The One that Sends Files Too Fast 


Business users of one of the applications that sends a file to a destination system ask 
the BizTalk administrator if itis possible to send the fles slower, like 20 messages per 
minute. The system that receives the files needs to consume a service (per file) that takes 





hours to complete and cannot handle more than a few files per minute, because itis 
‘legacy system that cannot be scaled or changed as they do not own the source code 

‘The BizTalk administrator automatically thinks of the maxConnection setting in the 
BizTalk Server configuration file, but then she realizes that the setting applies only to 
‘communications based on HTTP based adapters and, in this scenario, will not have any 
effect because the application is sending the files using the FILE adapter. 

She wonders now iby adjusting the host throtling settings, they can reduce the 
lrequency of the sending operations. She contacts then the developer team and explains 
the situation. The developer lead states that they can modify the rate-based throttling 
settings to delay the message publication or the message delivery, In both cases, the 
destination system will receive messages in a slower fashion. 

Em 
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"The BizTalk administrator suggests that Is better to slow down message publication. 
In this scenario, the fles will not be stored to the MessageBox because they would. 
remain in the file system, 

"The developer team agrees on that argument and starts to test the application with, 
different host throttling configurations. After a few attempts, they get to a configuration 
Where the BizTalk Server is picking up messages from the source folder at a rate ofone 
every three seconds. 


+ Minimum number of samples—Default value: 100, changed to 1. 
By changing to this value, the host throttling algorithm will start the 
throttling condition in every sample (Instead of 100 occurrences). 


+ Sampling window duration— Default value: 15 seconds, changed 
to 1 second, By changing this setting now every sample will last only 
one second. The throttling algorithm will consider the throuling 
condition every second. 


+ Rate overdrive factor—Default value: 12555, changed to 100%, By 
changing this threshold to 100%, the throtling condition will be 
raised when the number of delivered and published messages are the 


‘+ Maximum throttling delay—Default value: 300 seconds; changed to. 
33 seconde, The BizTalk Server engine will induce a delay of 3 seconds 
for every message received through this port. 


+ Throttling override Default value: Do not override, change toa 
Initiate throntling condition. The throttling condition will always rise 
ithe rate overdrive factor is reached. 


However, this configuration requires you to add a new host to the environment 
because they need to isolate the host settings for this specific receive port, 

"The BizTalk Server administrator approves all the changes and decides that a 
production assessment is not required because this host will process messages as per the 
requirement now, 

"The application goes live, and the destination system starts receiving around 20 
messages per minute, 
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Chapter 13: The One That Floods a Destination 
System 


Business users start complaining about the performance of one of the web applications. 
‘They say that during peak times, the application does not return the data and they get 
ots of timeout errors, 

‘The BizTalk administrator starts performance logging during the high load tim 
between BAM and 10AM. Once the logging finishes, she opens the result performance 
monitor BLG fle and starts analyzing the information. Upon examination, she notices 
that the following performance counters behave abnormally: 

+ BizTalk: Messaging Latency Outbound Adapter Latency (sec). 
Latency increases exponentially between 8 and 10AM. 
+ BizTalk: Messaging— Documents suspended/Sec. The number of 
suspended messages per second increases during the whole period 
JO messages. 





She opens the BizTalk administration console and filters suspended messages by 
destination URL She notices that all the suspended messages are related to the same 
destination, a Web Service that is hosted internally by a different team. She contacts 
the administrator of that service and together they realize that the server that hosts that 
Web Service shows 100% CPU and itis out of memory. The administrat 
‘checks the IS logs and he sees thousands of requests wi 

‘The BizTalk administrator suggests that it seems that the server is nor able to deal 
‘with dhe load that BizTalk Server is sending. So, she decides to adjust the maxConnection 
setting for that URL 

She opens the BizTalk Serv 








ofthat system 








‘configuration and checks the maxConnection setting: 


<systen.net> 
<connectiontanagenent> 
<add adáress«"*" maxconnections"25" /» 
«/connectiontanagenent» 
</system.net> 


She notices that every BizTalk Server destination is sending 25 messages, and that is 
probably the reason the destination system is loaded. She changes the maxConnection. 
setting to open only two connections to the affected service: 
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<systen.net> 
<connectiontanagenent> 
«add address«"*" maxconnection="25" /» 
add adéress-"http://ACHBlebService" maxconnection="2" /» 
c/connectiontanagenent» 
«[systen.net» 








"The day after, business users say that the sys 
anymore and the administration of the Web Service confirms that u 
to normal during the whole day 


n is not generating timeout errors 
PU and memory 








usage went back 


Chapter 14: The One with the New Application (Il) 


The 
implement a new low latency application that needs to process 100 messages per 
second, with peaks of 200 messages. 

As part of the application lifecycle, the development team introduced performance 








ME integration environment is growing. There is a new requin 


testing. The development team tests the solution in the testing environment and they ge 
the values shown in Table 10-2 for the performance counters 





Table 10-2. Performance Counter Data 








Performance Counter. Value 
Processor: Processor Time E 

Spool Size No increasing trends detected 
Available Memory * 2808 

‘SOL Server Disk ide Time 68% 

‘SOL Processor: Processor Time 25% 

BizTalk Server Disk dle Time 75% 

Documents Received [Sec 24 

Documents Processes [Sec 420 

Documents Sent /Sec 24 





"Note, the production environment has 30 GB of physical memory: 
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A in all it seems that the application can perform under the agreed performance 
SLA with the hardware that is used In the testing and production environments 
However, this test was executed while the testing servers were idle because the other 
applications were not tested at the same time, For this re lopment team. 
asks the BizTalk administrator to assess the production environment, so they can 
‘evaluate lf there is enough room to run the new application on production. 

‘The BizTalk administrator starts the performance logging for a period of 24 hours 
The day after, he processes the output BLG file using the PAL tool and gets the values. 
shown in Table 10-3 for the selected performance counters. 








won, the des 





Table 10-3. New Performance Counters Data 








Performance Counter Value 
"BizTalk Processor: % Processor Time as 

Spool Size No increasing rends detected 
Available Memory * 1068 

SOL Server Disk ide Time. 58% 

‘SQL Processor: Processor Time 51% 

BizTalk Server Disk Idle Time 45% 

Documents Received /Sec 355 

Documents Processes /Sec 598 

Documents Sent /Sec D 





"Note, the production environment has 30 GB of physical memory. 


"The ACME team compares the results to estimate the maximum resource utilization in 
production, if they deploy the new application. The comparison is shown in Table 10-4. 
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Table 10-4. Performance Counters Data Comparison 











Performance Count Testing Production Estimation 
BizTalk Processor: % ProcessorTime 32% E T 
Spool Size No increasing trends No increasing trend Unknown 
detected detected 
Available Memory 2868 1068 aca 
‘SOL Server Disk ide Time 68% 58% 25% 
‘SOL Processor: Y% Processor Time 28% 51% T 
BizTalk Server Disk dle Time. 75% 45% 15% 
Documents Received /Sec 24 355 EI 
Documents Processes [Sec 420 son 1018 
Documents Sent /Sec 24 D 686 





Based on this comparison, the team comes to the conclusion that adding this 
application to the platform would not be safe, as most of the estimated calculations show 
that there will be a good chance that th ent would start processing messages 
above the MST continuously. Both BizTalk and the SQL Server show high CPU and 
memory consumption. The whole team gathers together to discuss the current situation, 








with these conclusions: 


+ The performance SLA for the new application has been established 
10 100 messages per second, with peaks of 200. The team designed 
the load tests to reach the maximum load and the testing platform 
responded smoothly to the worst load scenario, showing that it ean 
process up to 244 messages per second continuously. Would it be 
acceptable to the Integrated parties if BizTalk Server slows down the 
application under high load? 


+ Ifthe integrated parties accept this, they can tune the BizTalk 
Server application hosts to enter the throtiling state for memory 
consumption when the number of received messages per second is 
higher than 110. This situation that will not arise frequently 


‘+ Mihis is not acceptable, then they will need to scale the environment 
byadding more hardware resources. 
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The integrated parties confirm that it would be acceptable ta slow dawn processing 
under high load. Now the question is, how do they get the right memory threshold? 
The BizTalk Server administrator suggests the following procedure: 


1. Isolate the receiving location into a separated host, 


2. Design a load test that will send 150 messages per second for one 
hour 


3. Gather the following performance counters 
^o BizTalk: MessageAgent: Process Memory Usage (MB) 
^o BizTalk Messaging: Documents received/see 

4. Run the load test. 


5, Analyze the Process Memory Usage (MB) counter during the full 
capture and perform an average calculation. This is the threshold 
that should be used to tune the host. So, when host instances 
consume this amount of memory, itis because the received 
documents per second is around 150. 


The whole team loved this idea and proceed as suggested by the BizTalk 
administrator. Upon testing, they got an average value for the Process Memory Usage 
(MB) counter of 2.166 MB (around 2 GB of memory). That means that the host instances. 
are consuming 6.6% of the available memory of the server (30 GB), 

Using the BizTalk administration console, they set the Process Virtual threshold 
ofthe receiving host to 7 and they perform the original load test again, As the host 
‘enters the throttling state due to memory pressure, messages take longer to complete, 
‘but the processor and memory utilization of BizTalk and the SQL Servers decrease 
exponentially. 

"They approve all the changes and the application is deployed to production. The 
BizTalk administrator performs daily performance assessments to production and 
‘confirms that the platform is still running under the MST. 
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Chapter 15: The One with the High Throughput 
Application 


‘A new application is ready for production. The development team has tested the solution 
‘on the testing environment and came to the conclusion that there is room for this 
application in production, The deployment is done aver the weekend and everything 
‘works normally during the initial integration tests. 

On Monday though, users complain about slowness of the platfor 
internal ticket. 

"The BizTalk administrator runs the BizTalk health monitor tool and finds out that 
the transaction log for the MessageBox database is around 40 GB (with actual data). He 
sends that information to the DBA and the SQL Server administrator confirms that the 
transaction log file for the MessageBox is growing to 40 GB. He recommends configuring 
the BizTalk Back Up so that the jobs run every five minutes, rather than leaving the 





nd open an 


default 15 minutes configuration. The BizTalk administrator approves the change and, 
upon testing finds out that withthe new job configuration, the transaction log for the 
MessageBox does not grow beyond 15 GB. 





Summary 


"This chapter went through several BizTalk Server problems that frequently occur 
around the world. Some of them were very simple and were fixed just by adding more 
hardware resources to the platform. However, in my experience, itis usually better to 
spend time analyzing the issue to find the root cause and use the flexibility that BizTalk 
Server provides to customize the platform. Eventually, especially if the number of new 
applications is growing, you will face a situation where the only solution is to scale the 








platform. But because you have been squeezing the platform settings to the maximum, 
your BizTalk Server environment will use resources efficiently. 
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physical disk:idle time counter, 458 
Spool Size counter, 457 
testing environment, 454 
WCF (see WCF service) 
Performance tools 
LoadGen, 109 
PAL, 109, 126 
performance monitor, 109 
Physical disk, 65 
Pipeline components 
assembling, 338 
attributes, 341 
custom, general, 341 
custom, properties, 344 
custom, Visual Studio Project 
(see Visual Studio project) 
disassembling, 338 
general, 337 
improving 
default passthru pipelines, 334 
Execute method, 334 
IDisposable interface, 334 
IPipelineContext object, 334 
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pipeline components, developing 
(see Pipeline components) 
XMLDocument object, 335-336 
XMLTextReader, 335 
interfaces, 339-340 
in receive pipelines, 396, 
in send pipelines, 336 
schema project, 342-343 
Visual Studio toolbox, 336 
Platform health alarm, 409 
Predicates, 27 
Privileged performance counter section, 60 
Processor time, 61 
PropertyGeneratorPipeline, 345-346 
Property schemas, 13 
Publish and subscribe model, 12 
Publishers, 38 
process, 39 
types, 38 





R 

Rehydration, 318 

Release stage level (RSL), 148. 
Request response, 32 
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Schema, improving. 
distinguished properties, 313-314. 
element length, 312 





property location, 313 
property name length, 313 
retrieve messages, 311 
routing, 313 
XPath expressions, 312 

SecondsEmptyBeloreShutdown, 326 
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SecondsldleleforeShutdown, 326 instance 
Send notification SMS, 404 orchestrations, 32 
Service level agreement (SLA), 81, 152 priority, 33-34 
Side-by-Side versioning request-response port, 32 
CLR engine, 368 solicit response port, 33 
component, 369 predicates, 27, 29 
solution searching information, 35-36, 38 
BookOrdersSolution, 370 System counters, 54 
assembly, fle versions, 373 
CRMOrderState element, 371 
maps section, 374 T 
orchestrations, 375-379 Target namespace, 6 
OrderStatus element, 371 ‘Testing, BizTalk Server solutions 
OrderStatus map link, 372 performance testing 
performance counters, 76 See Performance testing) 
resources section, 374 ‘unit testing (see Unit testing) 
schemas section, 375 Testing code 
{argetnamespace property, 380 act section, 438 
Side-by-side versioning technique, 367 arrange section, 438 
Solicit response, 32 assert object, 439 
Spool table, 18, 47-49 Assert. Faill), 437 
Spool performance counter, 54 assert section, 438 
Single Sing-On (S80) database, 367 Invalid amount, 439, 441 
add new keys, 390-391 returnTotalAmount method, 438-439 
assembly Binding Redirect, 389 test explorer, 441-442 
helper component, 394-395, 397 valid amount, 438 
export configuration, 392-393 ‘Threshold monitoring, 399 
import configuration, 393-394 ‘Threshold violation option, 404 
MMC Snap-In tool, 388 ‘Throttling 
related values, 392 conditions, 85 
retrieving configuration values, 396 database, 84,87 
setting value for the key, 391 defined, 84 
storage setting, 387 delivering rates, 84, 87 
Subscribers, 40-41 dequeuing, 84 
Subscriptions, 27,33, 35 Instance count, 87 
activation memory, 84, 88 
orchestration instance, 31 message agent, 87 
send port filter, 30 Message Box, 85 
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message count, 87 seal time, 266-267 
message delivery throttling -Stop command, 269 
state counter trace sessions, 266, 
dequeue process, 88 Visual Studio, 265 
global physical memory Windows 10 SDK, 266 
in-process queue, 90-91 ‘Tracking BizTalk performance 
message publishing rate, 90 BiTalkDTADb, 172 
NET CLR threads, 93-94 database, 171-174 
overdrive factor, 89 data per minute, 166 
rate-based, 95 disk contention issue, 171 
threads threshold, 93-94 dtasp_BackUpAndPurge 
Virtual memory, 92 TrackingDatabase, 172 
message publication, 84 diasp. PurgeTrackingDatabase, 172-173 
message publishing throttling host dedicate, 170-171 
state counter. log files, 171 
database size, 98-99 Message Box databases, 171 
faster, condition, 96-97 message size, 165 
Message Box database, 97-98 orchestration level, options, 167 
message count, 99 pipelines, options, 169 
publishing process, 95 process, 166. 
rate overdrive factor, 95-96 receive port level, options, 168 
Spool table, 100 RSlevel 170 
system memory, 97 schemas, options, 169 
threads, 100 sending port level, options, 168, 
user override, 100 SQLServer, 171 
virtual memory, 97 SQL Server query, 173-174 
messages, 84 ‘Tracking profile editor tool (TPE) tool, 287 
physical memor activity definition data, 293 
process memory, 88 approved element, 294 
publishing rate, 84, 87 BAM activity definition, 287-288 
severity customization, 86 BAM data, 295, 
thread count, 84, 88 BookOrdersApprovals 
TraceLog activity definition, 288 
buffered, 266, BookOrdersSolution. 
CAT Framework, 266 BizTalk application, 287 
event source, 289-290 
non-real time, 268 message payload schema, 293-294 
parameters, 267-268 MSI, 287 
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‘Tracking profile editor tol (TPE) tool cont) FReceivePipelineUnitTest() 
orchestration, 290-291. method, 451, 453 
query, 295-296 TestPipeline method, 451 
request response, 292 validate schemas 
"Transaction leves (TL), 150-151, 180 invalid instance, 446-447 
"Transaction type. IncomingBookOrders, 446-447 
property, 316-317 Validatelnstance method, 45 
valid instance, 445-446 
u 
Unit testing v 
BizUnit, 434 Visual Studio project 
BookOrders solution BaseCustomlTypeDescriptor 351 
BizTalk solution, 444 create test routing application 
interfaces, 444 custompipelineTesting 
Net Framework, 444 pipeline, 364-365 
references, 444 filter expression, 366 
Visual Studio, 43, 445 pipeline designer, 362- 
code testing (se Testing code) project properties, 363 
create, NET components TestSchema.xsd, 363 
BooksOrdersHelperTests class, 437 XML file, 366 
create unit tests dialog box, 436 custom, pipeline component, 360-361, 
TestMethod() attribute 437 Execute method, 354-355 
unit tests option, 435 — interface, 352-353 
Visual Studio, 435 aseMessageContest, 
maps a34 methods, 355-356 
NET components, 434 IComponent interface, 354-355 
orchestration, 434 IComponentlL interface, 359-360 
pipelines, a34 IniNew method, 354 
mas, 434 IPersistPropertyBag interface 
test menu, 453 GetClasslD method, 358 
test result, 453-454 GUID, 358 
validate maps Load method, 358 
TestMap method, 147 Read method, 356 
ValidinstanceMap method, 448 ReadPropertyBag, 356 
WronglastanceMap method, Save method, 358 
449-450 Write method, 356 
validate pipelines WritePropertyBag, 357 
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namespace class, 147 
pipeline interfaces, 344 
pipeline component, toolbox, 361-362 
PropertyName property, 351 
add image, 349 
BaseCustomTypeDescriptor, 352 
image designer tool, 350 
pipeline properties, 348 
string resources, 348 
steps, 345-347 
validate method, 359 
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WCE service 
header option, 489 
header properties, 490 
response, 492. 
run, test, 492 
SLA, 487 
SoapAction, 490 
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string body properties, 491 
URL property, 489 
Visual Ce project, 488 
WCF -RasicHup receive adapter, 487 
WebServiceBookOrders, 487 
Web Service request option, 488 
XML, 491 
Web service, 507 
Windows Management 
Instrumentation 
(wan), 102, 190-193 


X,Y,Z 
XML 
attribute, 2-3 
namespaces 
custom, 6 
default, 5-6 
Message type, 7 
schema, 2-4, 10-11 
XSL transform property 





